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ABSTRACT 

The combined field tests for the third follow-up 
survey of High School and Beyond (HSB) and the fifth follow-up survey 
of the National Longitudinal Study of the H*?h School Class of 1972 
(NLS-72) were conducted during the spring c£*1985. The overall 
purposes of the field tests were to evaluate questionnaires and 
procedures to be used for the full-scale surveys scheduled for 1986. 
Both the NLS-72 and the HSB questionnaires were reviewed for item 
nonresponse for all items with four percent or more missing data. 
Both questionnaires included a large number of skip patterns. 
Empirical results concerning response rates to skip patterns were 
discussed and special problems requiring revision were analyzed. A 
calendar format was tested to offer an effective way A Cc^turing 
periods of interrupted schooling and employment, reducing the size of 
the main survey data file rithout compromising data quality. The 
effectiveness of new and revised HSB and NLS-72 items were evaluated 
by frequency distributions and evaluators' observations. The current 
follow-up surveys of HSB and NLS-72 were conducted primarily by mail. 
However, locating for the NLS-72 field test was done by telephone. In 
addition, a separate questionnaire, the Teaching Supplement was 
developed for a subset of the NLS-72 sample. (PN) 
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1 . FIELD TEST ACTIVITIES 

The combined field tests for the third follow-up survey of 
High School and Beyond (HS&B) and the fifth follow-. up survey of 
the National Longitudinal Study of the High School Class of 1972 
(NLS-72) were conducted during the spring of 1985- Both HS&B and 
NLS-72 are longitudinal studies sponsored by the National Center 
for Education Statistics (NCES) of the Department of Education. 
The overall purpose of the field test was to evaluate 
questionnaires and procedures to be used for the full-scale 
surveys scheduled for 1986. 

Since the surveys are longitudinal, most of +,he questionnaire 
items used in the field tests have appeared in previous waves. A 
few new items were added to HS&B; and many new items were added to 
NLS-72, especially in relation to the teaching profession. In 
addition, a separate questionnaire, the Teaching Supplement, was 
developed for a subset of the NLS-72 sample. The questions on 
teaching and the Teaching Supplement were supported by a National 
Science Foundation (NSF) contract. 

Survey materials were submitted to the Federal Education Data 
Acquisition Council (FEDAC) in November 1984. Minor revisions 
requested by FEDAC were made, and the packages were submitted to 
the Office of Management and Budget (OMB). Approval was granted 
in January 1985. 

Although the current rounds of HS&B and NLS-72 will be 
conducted in tandem, they were begun approximately eight years 
apart. There are many similarities between the two studies; 
however, each has its unique characteristics and history. 
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HS&B began in 1980 with a base year collection of test and 
survey data from approximately 30,000 high school sophomores and 
28,000 seniors. Since that time, two follow-up surveys have 
occurred—one in 1982 and the other in 1984. For che first 
follow-up survey, the senior cohort was subsampled and has since 
remained constant. The sophomore cohort was subsampled at the 
time of the second follow-up survey. Hence, the third follow-up 
f*mple consists of 11,995 members of the senior cohort and 14,825 
members of the sophomore cohort. 

In preparation for the full-scale third follow-up survey, a 
field test was conducted with 4-08 individuals who were selected 
from an independent sample that was drawn for the base year field 
test in 1979. These young adults were sophomores and seniors at 
the time of the field test for the base year survey. 

The primary objective of the HS&B third follow-up field test 
was to evaluate new and revised survey items. Although the 
questionnaire core was well established, the third follow-up 
survey will be the first time the same instrument will be used for 
both the sophomore and senior cohorts. Thus the applicability of 
questions to both cohorts, as well as the performance of new and 
revised items, was of interest during the field test. 

The schedule and procedures for the full-scale third follow- 
up survey in 1986 will be the same as those used during the second 
follow-up survey. Therefore, there was no reason to alter the 
schedule for the current field test. Survey members were located 
during the fall of 1 984, data collection began in February 1985, 
and data preparation was completed by June 1985. 



During the NLS-72 base year, test and survey data fron ?3,000 
high school seniors were collected. The sample for the fifth 
follow-up survey scheduled for 1966 involves close to 14,500 
members of the original sample. Approximately 700 individuals 
were selected from the main fifth follow-up sample to participate 
in a field test prior to the full-scale survey. 

The general purpose of the field test for the fifth follow-up 
survey of NLS-72, as for HS&B, was the evaluation of survey 
instruments and procedures. However, due to the long time lapse 
since the fourth follow-up and the emergence of new analytic 
issues, there were a greater number of specific objectives. 
One major objective was to assess the scope of locating and 
persuasion tasks that would be needed to ensure a satisfactory 
response rate of the NLS-72 sample. Although there had been two 
address updating efforts (in 1981 and 1983), a thorough tracing of 
unlocated or nonresponding survey members had not been undertaken 
since the fourth follow-up survey in 1979. Furthermore, since 
there had been an extended interval between data collection waves 
(the cohort had noz been surveyed since 1 979 ) » the willingness of 
the fifth follow-up sample members to participate was 
unpredictable . 

In contrast to the ESL3 questionnaire, the NLS-72 instrument 
had been modified extensivly, thus requiring a considerable amount 
of instrument assessment. (See the section on new NLS-72 
questionnaire items.) Some questionnaire items were modified to 
gather data for a relatively long period of time (six years) since 
the last follow-up survey. Modifications also were made to 
facilitate the study of the current life cycle period of the NLS- 
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72 cohort. Finally, revisions were made to permit comparative 
analyses of the NLS-72 sample and the HS&B cohorts. Many NLS-72 
items and formats were changed to conform to the HS&B 
questionnaire. 

The Teaching Supplement is an entirely new set of questions 
for the NLS-72 cohort* This supplementary questionnaire was 
developed to address topics such as teacher quality, career 
motivation, teaching experiences, and career satisfaction. 

The rest of Section 1 describes the f: :Id test 
questionnaires, the samples, data collection, and data preparation 
tasks. Field test procedures and their results are reviewed and 
discussed with attention to their implications for the full-scale 
surveys in 1986. 

Analyses of field test data are presented in subsequent 
sections of this report. Each section is designed to discuss a 
specific objective of the field test and to yield recommendations 
for modification of questionnaires and survey procedures. 

1.1 Survey Instruments 

A principal field te3t objective for both HS&B and NLS-72 was 
the evaluation of questionnaire performance to inform revisions of 
the instruments for the full-scale survey. The field test 
provided an opportunity to test questionnaire items that had been 
revised substantially since the last follow-up survey and to gauge 
the performance of items that were completely new. The 
questionnaires, therefore, included more items than were planned 
for the full-scale survey. 



Evaluation of the field test instruments was based on 
analyses of field test data, on respondents' comments (both 
written and offered during telephone follow-up), and on a thorough 
review of completed questionnaires by the NORC coding staff. One 
of the goals of the evaluation process was to make recommendations 
about minimizing the burden imposed on respondents by lengthy 
questionnaires and, specifically, to recommend questionnaire items 
to be eliminated during the finalization of the survey documents. 
These recommendations, supported by discussions of issues related 
to questionnaire performance such as response variation, item 
nonresponse, and respondent burden, are addressed in other 
sections of this report. 

This section reviews the current structure and contents of 
the HS&3 questionnaire, and the NLS-72 core questionnaire and 
Teaching Supplement. 

1.1 Survey Instruments 

A principal field test objective for both HS&B and NLS-72 was 
the evaluation of questionnaire performance to inform revisions of 
the instruments for the full-scale survey. The field test 
provided an opportunity to test questionnaire items that had been 
revised substantially since the last follow-up survey and to gauge 
the performance of items that were completely new. The 
questionnaires, therefore, included mor. items than were planned 
for the full-scale survey. 

Evaluation of the field test instruments was based on 
analyses of field test data, on respondents* comments (both 
written and offered during telephone follow-up), and on a thorough 
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review of completed questionnaires by the NORC coding staff. One 
of the goals of the evaluation process was to make recommendations 
about minimizing the burden imposed on respondents by lengthy 
questionnaires and, specifically, to recommend questionnaire items 
to be eliminated during the finalization of the survey documents. 
These recommendations, supported by discussions of issues related 
to questionnaire performance such as response variation, item 
nonresponse, and respondent burden, are addressed in other 
sections of this report. 

This section reviews the current structure and contents of 
the HS&B questionnaire, and the NLS-72 core questionnaire and 
Teaching Supplement. 

1.1.1 HS&B Core Questionnaire 

Due to the longitudinal nature of the study, a core of HS&B 
questionnaire items have remained mostly the same from one 
follow-up to the next. Yet with each wave a unique set of 
questions has been added to provide information that nas current 
policy relevance. For instance, the first follow-up survey 
included questions on drug knowledge and use, and the second 
follow-up survey contained questions on familiarity with 
computers. 

The HSftB third follow-up questionnaire currently include', the 

following sections which cover the topics indicated. 

Background Information: Respondent's birthdate, 
current activities, household composition, and 
geographic proximity to his/herhigh school. 

Work Experience: Information on up to four jobs, 
salary, how jobs were found and reasons for leaving; 
job training, employer-related training benefits and 
programs; attitudes toward current job, iob 



satisfaction; expectations about future careers or 
employment situations. 



Periods Unemployed: Periods of unemployment, time 
devoted to looking for work, methods used for job 
searches . 

Education: If and when respondent completed high 
school; educational aspirations; information on up to 
two postsecondary schools; attitudes toward personal 
educational experiences; and school finances. 

Other Training: Participation in apprenticeships , 
manpower training programs; noncredit educational 
activities. 

Family Information: Current marital status; marital 
history; occupation and educational background of 
spouse; anticipated family size; information on up to 
seven children. 

Income: Sources and amounts of respondent income. 

Experiences and Opinions: Attitudes toward financial 
matters, sex, roles, and self; political and voting 
behavior, participation in voluntary organizations ; 
leisure activities; importance of family, friends, 
work, and other aspects of personal life; physical and 
psychological disabilities. 

School Record Inf creation: Permission form to release 
postsecondary school records for survey purposes; 
request for social security number. 



1.1.2 NLS-72 Core Questionnaire 

The NLS-72 questionnaire has been revised extensively. Many 
questions appeared in previous NLS-72 follow-ups and constitute 
the "core" of the instrument. Some of these items have been 
modified only slightly (e.g., references to years were updated and 
r.inor wording changes made). Other items have been revised more 
substantially with major changes in content or format. New 
questions were added to the core to gather information on job 
training and child care. The new and significantly modified items 
are reviewed in detail in the sections on "New Questionnaire 
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Items--NLS-72 Core Questionnaire" and "Event History Item 
Formatting (Calendars)" sections of this report. 

Substantial additions to the questionnaire were sponsored by 
the National Institute of Child Heath and Human Development 
(NICHD). Included were items that ask about marriages and 
ma: riage-like relationships, with special focus on first 
marriages, separations, and divorces. Also included were items 
that gather event histories on respondents' fertility, child-care 
responsibilities, and behavior of noncustodial parents. 
Questionnaire items that covered flows of resources between 
respondents and individuals outride the respondent's household and 
intergenerational transfers of resources were included as well. 
Most of the NICHD questions are in the Family Information and 
Household and Parents sections of the questionnaire (see list 
below) . 

Other substantial additions to the questionnaire were 
sponsored by the National Science Foundation (NSF). These items 
were included to investigate aspects of the teaching profession 
that have encouraged or discouraged the pursuit of teaching as a 
career. They are the final series of questions in the instrument 
and are referred to on the list below as Teaching Interest and 
Attitudes . 

The NLS-72 fifth follow-up questionnaire currently includes 

the following sections and covers the topics indicated. 

General Information: Respondent ' s birthdate; current 
activities. 

Work Experience: Information about up to four jobs, 
salary, how jobs were found and reasons for leaving; 
employer-related job training benefits and programs; 
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attitudes toward current job, Job satisfaction; periods 
of unemployment, 'inemployment insurance. 



Education: Information about up to three postsecondary 
schools attended; current level of education; 
educational aspirations, field of study; school costs, 
resources, and financial aid; financial assistance from 
parents, dependent/nondependent income tax status. 

Family Information: Marital history and family 
formation with an emphasis on the first marriage; 
divorce settlement, child custody and support, 
attitudes toward former spouse, post-divorce standard 
of living; current marital status, spouse's occupation, 
educational background, and current activities; 
amicipated family size; information for up to seven 
children. 

Child Care: Arrangements for preschool and school-aged 
children, costs; educational expectations for children, 
financial planning for children's education. 

Households and Parents: Past and current composition 
of respondent *s household; household financial 

arrangements; social and economic networks and 

resource flows; parents' age, proximit3' to respondent, 
frequency of interaction, respondent involvement in 
planning for parents' retirement. 

Background Information: Ethnicity, race, social 
security number; voting behavior. 

Income: Sources and amounts of respondent's income. 

Teaching Interest and Attitudes: Motivating and 
dissuading factors in teaching career decisions; 
opinions about teach * quality. 



1.1.3 Teaching Supplement 

A Teaching Supplement questionnaire was also sponsored by 
NSF. The instrument, which accompanied the main NLS-72 
questionnaire, was designed to explore the career-related 
attitudes and experiences of survey members who are teache/s, who 
were teachers, or who had prepared to be teachers. 

The sections oi* the Teaching Supplement for the NLS-72 fifth 
follow-up survey and the topics covered are as follows. 
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Certification: Requirements of teacher training 
program and certification; level and field of 
certif icaticn. 



Actual Teaching Experience: Periods of teaching, 
subject areas and grade levels; college courses taken 
related to areas and levels taught; respondent's self- 
assessnrnt of preparation to teach; ability of 
students; professional development; retraining for 
another teaching field. 

Allocation oiT Time: Classroom teaching, non-classroom 
duties, classroom activities. 

Satisfaction with Teaching: General satisfaction; 
expectations about duration oi' teaching career. 

School: Size of teaching znd professional j^aff; 
assessment of student economic stratum; s^ool 
location; respondent perception of change.. *n school 
activities, relationships, environment, and 
expectations; measures to improve student achievement; 
requirements for graduation (high school); reasons not 
employed as a teacher; collective bargaining, 

1 ,2 Sample Design 



1,2,1 High School and Beyond 

The sample, drawn for the base year fielC test, has been 
reused for each follow-up field test. It was augmented (at the 
time of the second follow-up field test) to ensure that it 
represented the variability in the main sample and in order to 
meet various field test objectives. The original structure and 
current status of the field test sample are described in this 
section. The original sample was designed to be representative of 
the population of U,S, high school students. It waj a purposive 
sample, drawn in stages. During the first stage, six states were 
chosen to represent the four census regions. At the second stage, 
a total of 4-2 schools was chosen from a stratification of schools 
within tne states. Finally, 36 students within schools were 
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selected from roster^ of the 1978-79 sophomore and senior classes 
using systemmatic sampling procedures. Approximately, 1,500 
students from each class were selected. Data were collected from 
1,291 sophomores aad 1,24-5 seniors during the base year field 
test. 

Although the first follow-up field test was not fully 
implemented, address update forms were sent to the 2,536 base year 
participants in 1 981 and returned by approximately 510 respondent* 
(265 sophomores and 24-5 seniors). 

The second follow-up field test sample consisted of the 510 
individuals for whom address updates were obtained in 1981, An 
analysis of the sample indicated a higher proportion of students 
with grades in the A-Ii range than in the base year field test 
sample. To reduce possible bias toward high ability respondents, 
the sample was augmented with 104- individuals selected from the 
base year field test sample who had reported grade averages of 69 
(D) or below, A second criterion for the selection of the 
augmentation cases v*- nonresponse to the address update request 
in 1981, The purpose of adding this group was to approximate 
locating difficultier that were anticipated for the main study. 
After sending out a locating letter, addresses were eventually 
verified or updates for 4-08 members of the 614-member sample. 

The current field test sample is composed of the 4-08 survey 
members for whom address updates were obtained during the second 
follow-up rield test. There were 207 members of the sophomore 
cohort and 201 members of the senior cohort. 
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1.2.1.1 Implications of Sample Characteristics for Interpreting 
Field Test Responses 

In order to assess the comparability of the field test sampla 
to the irair sample, field tost data from the education, 
employment, family, and background sections of the questionnaire 
were compared with data from the second follow-up survey (senior 
cohort). Comparison of behavioral and attitudinal measures 
reveals very little difference in the characteristics of the 
samples. It should be noted that field test data are unweighted, 
and therefore caution should be exercised in comparing the tvo 
groups in ways other than on general questionnaire performance. 

Education: On a measure of educational attainment 
(grad.iation from high school) the two samples were very similar. 
The percentage of the main sample that repo ted graduating from 
high school was 97.5 percent compared to 94- percent for the field 
test. 

The educational aspirations of the two groups were also 
similar. Of the field test respondents, 39 percent planned to 
finish two to four years of college compared to 4-1 percent of the 
main sample. Twenty-thre'. percent of the field test respondents 
aspired to master's degrees or doctorates compared to 18 percent 
of the main sample; and 15 percent of the field test sample 
planned to attend 2 to 4- years of vocational school compared to 18 
percent of the main sample. 

Employment: Members of the field test sample were only 
slightly more likely than the main sample to have experienced 
unemployment. Thirty-three percent of field test respondents 
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reported being unemployed at some time during the previous 21 
months compared to ';>0 percent of the main sample. 

The long-range employment expectations of both groups were 
similar. More respondents in both groups expected to be in 
professional occupation? at age 30 than in any other employment 
category. Those expecting to have managerial positions were just 
over 10 percent in both groups and those expecting to have 
clerical jobs were also just over 10 percent in both groups. 
Approximately 8 percent of both samples expected to be teachers. 

Family Information: Survey data indicated that the household 
compositions of the two groups were comparable, (Household 
composition is defined as the people with whom the respondent was 
living at the time specified, ) The main sample was slightly more 
likely to be living with a parent, grandparent, and/or siblings. 
The field test sample was more likely to be living with a spouse 
and/or children. A comparison of the marital status indicated 
that eight percent more field test than main sample participants 
were married. The two groups also expected about the same number 
cf children. The differences in expectations in family size did 
not exceed four percentage points. 

The small differences in household compositions and percent 
married are likely to be due to the small difference in the ages 
of the samples and in life experiences such leaving parents 1 
home, marriage, and beginning families. Members of the field test 
sample were high school sophomores and seniors in 1979; the main 
sample was selected one year later and was therefore one year 
younger , 




Background Information: Forty-seven percent of the main 
sample respondents reported living in the same place as they did 
when they were in high school, whereas thirty-nine percent of the 
field test respondents reported living in the same area. However, 
the proportion of the main sample who lived within a 100-mile 
radius of their high schools (27 percent) was less than that of 
the field test sample (39 percent). 

This rudimentary comparison of field test and main survey 
samples suggests that the field test sample is similar enough to 
the main sample to permit reasonable assessment of questionnaires 
and survey procedures. 

1 .2.2 NLS-72 

The existing NLS-72 field test sample was not suitable for 
the fifth follow-up field test. It had been drawn in 1971, and 
contact ' th sample members had not been maintained over the 
intervening years. Locating and gaining the cooperation of sample 
members after an extended period of survey inactivity posed major 
problems. Furthermore, costs to locate members of this sample 
after a long period without contact would have been prohibitive. 
Based on these considerations, a new field test sample was drawn. 
It was taken fi-om the main sample selected for the fifth follow-up 
and will be retained as part of the main sample for the full-scale 
survey. A brief description of the main sample and the current 
field test sample are presented below. The fifth follow-up sample 
is a subsample of the estive NLS-72 population of 22,652 students. 
The active population consists of those persons who participated 
in at least one of the previous waves of NLS-72. 
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The sample for the fifth follow-up retains the basic design 
of the base year, which was a stratified two-stage probability 
sample. During the first sta^e, schools were selected from strata 
based on type of control (public or private ), geographic region, 
enrollment size, geographic proximity to institutions of higher 
education, proportion of minority group enrollment, community 
income level and degree of urbanization. During the second stage, 
students were selected on a probabilistic basis within schools. 

The fifth follow-up sample was designed so that several 
policy-relevant groups would be retained with certainty. These 
groups were Hispanics, teachers and potential teachers, persons 
with college or advanced degrees, persons who had experienced 
marital disruptions (divorced, widowed, separated), and never- 
married parents. The strategy for selecting the rest of the 
sample was based on tradeoffs between cost and sample efficiency. 
The remaining cases (those not selected with certainty) were 
cross-classified irto groups according to whether they had 
participated in the 1979 fourth follow-up survey (participants and 
nonpar ticipants) and by college experience (those who reported two 
years of college or less, and all others). The selection rate 
from these groups differed and was influenced primarily by 
analysis objectives, A total of 14-, 4-86 cases was selected. 

To select the field test sample, the fifth fo\low-up sample 
of 14-, 436 was partitioned into five strata, which are shown in 
Table 1 2,2-1, Strata were determined by a combination of the 
policy relevance of certain groups and by field test objectives 
(such as analysis of cost and effectiveness of survey procedures), 
A total of 697 cases were drawn from the main sample, 
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Table 1 .2.2-1 
NLS-72 Fifth Follow-Up Survey Field Test Sample 

Stratum Stratum Size Stratum Characteristics 



A 200 Teachers who were fourth follow- 

up participants* 

B 125 NICHD cases who were fourth 

follow-up participants but 
not teachers** 

C 275 Other fourth follow-up 

participants 

D 25 Fourth follow-up nonparticipants 

who were not NICHD cases 

E 72 NICHD cases who were fourth 

follow-up nonparticipants 



Total sample: 697 



*A M teacher M is a survey member who, according to information 
from the third and fourth follow-up surveys: (1) worked as a 
teacher at any level (except college or university); or (2) 
worked as an elementary or secondary school administrator; or 
(3) received a 2- or 3-year vocational degree, a 2-year 
academic degree, a 4— or 5-year college degree, or a master's 
degree in education; or (4-) received a 2- or 3-7°» y - vocational 
degree, a 2-year academic degree, a 4— or 5-year college degree, 
or a master's degree and indicated that they intended to be 
a teacher or thought that they would be a teacher by the time 
they were 30 years old; or (5) received a 2- or 3-year 
vocational degree, a 2-year academic degree, a 4— or 5-year 
college degree, or a master's degree in science, mathematics, or 
engineering. 

**An "NICHD case" is a divorced, widowed or separated spouse (or 
former spouse) or a person who was ever a never-married parent. 



1.2.2.1 Implications of Sample Characteristics for Interpreting 
Field Test Results 

In order to assess locating procedures and costs, the field 

test waa purposely loaded with NLS-72 survey members who did not 
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participate in the fourth follow-up. In the field test, this 
group comprised approximately 14 percent of the sample, compared 
to about 3 percent in the main sample. It was expected that it 
would be more difficult and expensive to locate fourth follow-up 
nonparticipants than those who had participated in the 197? 
survey. Ample representation of this group provided an 
opportunity to consider the effectiveness of various locating 
procedures and to estimate the level of effort and cost that will 
be required for locating nonparticipants during the full-scale 
survey in 1986. 

Results of the prefield locating activities supported 
assumptions about the difficulty in locating nonparticipants- 
Nonparticipants were located at less than half the rate of 
participants (see the section on "Repondent Locating"). Since 
locating problems ultimately affect survey completion rates, zhe 
over-representation of nonparticipants in the sample resulted had 
an adverse effect on the final overall completion rate (see Data 
Collection) . 

In interpreting field test data, the overrepresentation of 
nonparticipants may have increased item nonresponse. The 
participation histories of these survey members demonstrates they 
are less inclined to cooperate in survey activities. They might 
also be less likely to fill out questionnaires in a careful, 
thorough manner . 

One of the field test objectives for NLS-72 was to evaluate 
new questionnaire items about the teaching profession and to 
estimate the proportion of the main sample to target as 
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respondents for these items. During the field test, teachers and 
"potential" teachers were asked to answer these questions. 
Teachers were defined as those who had indicated, during previous 
follow-up surveys, that they had worked as twachers or 
administrators at any level other than in higher education, and 
those who had obtained education degrees. The definition of 
"potential" teachers was purposely broad in order to include as 
many individuals as possible who might at any time have had an 
interest in teaching. The goal was to include not only those who 
might consider teaching in the future, but also those who had 
considered teaching in the past and rejected the field. Potential 
teachers were defined as those who had received vocational 
degrees; those who had received two-year . four -year, or five-year 
college degrees; or those who had master's degrees and also had 
indicated interest in a teaching career during a previous follcv- 
up survey. Potential teachers also included survey members who 
had studied science, mathematics, or engineering in a 
postsecondary school. 

Teachers (as defined above) were over sampled for the field 
test to provide an optimal size group for evaluating the Teaching 
Supplement questionnaire items and for estimating the proportion 
of the sample cho would be appropriate respondents to answer them. 
Teachers constituted almost 30 percent of the field test sample 
but less than 17 percent of the main sample. Because the 
principal criterion for selecting teachers was level of education, 
the overreprefientation of teachers in the field test sample might 
have resulted in better respondent performance on the more 
difficult sections of questionnaires (e.g.. those' with complex 

18 

er|c 21 



skip instructions)- Such a bias, however, might have been 
balanced by the possible nonresponse effect of fourth follow-up 
nonpar tic ip ants discussed above. 

1.3 Data Collection and Preparation 

Data collection consisted of a prefield and a field period. 
The major activity during the prefield phase was locating survey 
members and updating name and address information. Prefield 
locating activities were conducted from September to December 
1 984. Pleas^ refer to "Respondent Locating" for a review of 
prefield procedures and an analysis of outcomes. 

The field period, vhich began in February and continued 
through May of 1985, was devoted to the simultaneous collection of 
survey data from the HSftB and NLC-72 samples. Obtaining completed 
questionnaires from at least 80 percent of each sample was the 
goal during the field period. 

In the first week of February, questionnaire packages 
were mailed to survey members whose addresses had been updated 
during the fall locati/ig period. Packages contained 
questionnaire(s) , a cover letter, a newsletter, a general 
information sheet, a $5 incentive check, a pencil and a return 
envelope. All survey material was mailed first class with Address 
Correction Requested specified on the envelopes. A Teaching 
Supplement questionnaire was included in the packages sent to a 
subsample of 200 NLS-72 cases identified as teachers or 
"potential" teachers. (See the "Sample Design" section of this 
report for a description of thn sampling procedures.) 
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Follow-up postcards were sent to survey members three weeks 
after questionnaires were mailed. The purpose of the postcard was 
to thank those who had completed and returned their questionnaires 
and to encourage those who had not yet returned their 
questionnaires to do so promptly. 

Telephone prompting began for those who had not responded 
after four weeks. Telephone interviewers calling from NORC's 
central office in Chicago contacted survey members to urge them to 
complete and return questionnaires. Offers to remail survey 
material were made to those who reported they had not received, 
had lost, or had misplaced questionnaires. Several rounds of 
reminder calls were made. Calls were staggered, so that contacts 
with respondents were approximately two weeks apart. 

The remainder of this section is a review of data collection 
results for the two samples. 

1.3.1 HS&B Data Collection Results 

In February 1985, questionnaire packages were mailed to 389 
of the 4-08 members of the HS&B field test sample. These were 
individuals whose addresses had been updated during the prefield 
period or for whom some mailing address was available. At the end 
of a 15-week data collection period, an overall response rate of 
81.9 percent had been achieved for the two cohorts. 

As Table 1.3.1-1 shows, almost half of the questionnaires 
(4-9.3 percent) were returned by the end of the third week of daxa 
collection. There was a substantial decrease in the rate cf 
return after the third week. The rate continued to decline with 
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Table 1 .3.1-1 



HS&B 

Weekly Completion Rate 
by 
Cohort 



S0PH0M0R2S 
(N=207) 



! 

! Main 
Field 
Test 
Sample 
(N«190) 



Column 
Totals: 



Augment 
(N=17) 



SENIORS 
(N=201 ) 



Main 

Field 

Test 

Jample 

(N=178) 



Augment 

(N=23) 



1 64 
(79.2*) 



170 
(84. 6%) 



















Week 1 


16 




25 


1 


42 


42 


10.3 


Week 2 


4-7 


3 


48 


1 


99 


141 


34.6 


Week 3 


31 




28 


1 


60 


201 


49.3 


Week 4 


15 




13 




28 


229 


56. 1 


Week 5 


16 


1 


7 


2 


26 


255 


62.5 


Week 6 


7 


2 


12 


2 


23 


278 


68. 1 


Week 7 


2 




1 


2 


5 


283 


69.4 


Week 8 


6 


1 


7 


1 


15 


298 


73.0 


Week 9 


3 




5 


1 


9 


307 


75.2 


Veek 10 


3 


i 5 
I 




8 


315 


77.2 


Week 11 


6 


1 


5 
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80. 1 


Week 12 ! 1 I 
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1 
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Week 15 I 3 
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3 
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81 .9 
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Totals: 1 (81.7*) (4-7.1*) 
I \ / 


157 13 
(88.2*) (56.5*) 
\ / 


334 
(81 .9*) 



Weekly 
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Cum 
Total 
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the exception of small upturns in Week 3 and Week 11, attributed 
to telephone prompting. 

The first round of telephone prompts began in early March, 
Week 5- The benefit of this telephone contact with survey members 
was realized approximately three weeks later durin£ Week 8. The 
second round of prompting calls began in late March, during Week 
7. An increase in questionnaire receipt occurred about four weeks 
after these calls began. For the second set of telephone prompts, 
there was a greater time lag between prompt calls and 
questionnaire receipt, and the rate of questionnaire return was 
not as significant as the rate that resulted from the first 
prompt. This is not surprising since the more difficult cases 
generally are dealt with at the end. Approximately 6 percent of 
all questionnaires that were eventually returned could be 
attributed to the initial telephone prompt; only 3.7 percent came 
in as the result of the second prompt. 

There was a relatively small difference in the completion 
rates for the sophomore and senior cohorts (79.2 percent and 84-. 6 
percent, respectively). There vers, however, significant 
differences between the completion rates of the main field test 
sample respondents and the augmentation cases. Survey members 
added to augment the sample were individuals who did not respond 
during the locating effort in 1981, Their nonresponse in 198i and 
low response rate during this field test demonstrate the 
reluctance of this group to participate. Gaining participation of 
main sample survey members with similar characteristics will 
require substantial resources. 
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Table 1*3.1-2 compares the current field test 
results with th- previous field test, and suggests possible trends 
that have implications for the data collection schedule and 
completion rate during the full-ncale survey in 1986. Fifteen 
weeks rather than 11 weeks were required for this field test to 
achieve a complexion rate comparable to that of the 1983 field 
test. During the full-scale survey, it may be wise to involve the 
field staff at an earlier point in order to achieve the desired 
completion. Although the NORC field staff is not involved in data 
collection during field tests, they are in place for the full- 
scale survey. They undertake intensive locating efforts for 
survey members not reached from the central office. Field staff 
also have responsibilty for telephone prompting and telephone and 
personal interviews. 

The completion rate of the two cohorts, which was almost the 
same in 1983. differed by almost 6 percentage points during this 
field test (see Table 1.3.1-1). The completion rate (84.6 
percent) for thw elder cohort was an improvement over the last 
field test (82.1 percent); the completion rate for the younger 
cohort (79 percent) was slightly lower than the last field test 
(81 percent). Although these are email differences, they suggest 
the possibility of a trend among the sophomores that warrants 
attentive monitoring during the full-scale survey. 
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Table 1 .3.1-2 



HS&B 

Comparison of Second and Third Follow, , r Field Test 
Weekly Mail Receipt of Completed Quest ionnaire& 
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1.3.2 NLS-72 Data Collection Results 

Although the overall response rate for the NLS-72 sample 
cohorts was 74-. 6 percent, there were considerable differences in 
the completion rates of the five subsamples (Stratum A through E). 
As Table 1.3.2-1 shows, the response rates of Stratum A and C were 
87.5 percent and 83.5 percent, but the rates for Stratum B, D, and 
E were much less favorable (65.6 percent, 4-7.7 percent, and 28.0 
percent, respectively). These low response rates are not 
surprising given the characteristics of the subsamples. Stratum B 
members were selected because they had had marital disruptions or 
were never-married parents. It is likely that a group chosen on 
the basis of such criteria would be more difficult to find and 
less inclined (as a group) to cooperate with the completion of a 
questionnaire than a group with more conventional patterns of 
family formation. Those in Stratum D and E were nonparticipants 
during the fourth follow-up survey. The low completion rates for 
these groups were attributable to considerable difficulty in 
locating them. Only 37 percent of the survey members in these 
strata were found during the fall locating period. 

If the NLS-72 mail receipt pattern is compared to that for 
HSAB, different curves in response rates are apparent. In 
general, NLS-72 survey members returned questionnaires at a slower 
rate than the HS&B cohorts* This war true even for the two hign 
completion rate strata (A and C). Approximately half of the HS&B 
questionnaires were returned after three weeks, but it was not 
until Week 5 that the Stratum A and C subsamples had achieved 
similar results, and it was not until Week 8 that a comparable 
rate u ,ad been reached by Stratum B. The completion rates for 
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Table 1 .3.2-1 



NLS-72 
Weekly Completion Rate 
by 
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♦Stratum A: Teachers who were fourth follow-up participants 
Stratum B: NICHD cases who were fourth follow-up participants 

but not teachers 
Stratum C: Other fourth follow-up participants 

Stratum D: Fourth follow-up participants who were not NICHD cases 
Stratum E: NICHD cases who were fourth follow-up 
nonparticipants 
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Stratum D and E never reached 50 percent. All of this suggests 
that the NLS-72 sample will require a greacer level of effort in 
order to attain desired response rates. 

It is important to recognize that the response rate for the 
field test was heavily influenced by the NICHD supplemental 
sample. The response rate for the NICHD cases was 50,5 percent, 
while the response rate for the NCES sample was 81.9 percent. 
Since 197 of the 697 cases were NICHD samplt members, their lower 
response rate ha- a strong impact on the overall field test 
response rate. 

Field test results have implications for completion rate 
goals and allocation of data collection resources during the main 
study in 1986, The very different response rates of the strata, 
given approximately the same level of effort, suggest that it will 
be necessary to establish some priorities in the effort expended 
on each of the strata during the full-scale survey. For 
instance, although some improvement of the poor completion rate of 
Stratum E (28,0 percent) is possible, achieving a rate comparable 
to the other strata is not likely. The Stratum E response rate is 
th3 result in large measure of locating difficulty -a 
characteristic of the stratum because of sample selection criteria 
(i.e., Stratum E cases are nonparticipants as well as NICHD 
cases). Therefore, data collection efforts and costs past a 
certain point are not likely to be productive. The recommended 
alternative to directing extra effort to increasing the Stratum E 
response rate is giving priority to Stratum B (also NICHD cases 
but offering less locating difficulty because they were fourth 
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follow-up participants)- Such prioritization would maximize data 
collection resources- Of course, if unanticipated trends become 
apparent during the full-scale survey, priorities can be 
readjusted. 

Follow-up calls were made ty NORC telephone interviewers 
during the data collection period to encourage survey members to 
return questionnaires. Some of those contacted indicated they 
would return questionnaires bi.t ultimately failed to do so. When 
the field period had ended, these individuals were recontacted and 
asked why they had not returned questionnaires. They were also 
asked, if anything might have been done differently to secure their 
cooperation. 

The reason survey members most frequently gave for not 
returning questionnaires was that they lacked the time to fiJ 1 
them out. An extension of calendar time would not have been a 
solution. Responders felt their personal daily schedules were 
already too full to add what appeared to be a time-consuming task. 
Most other reasons were also related to burden: the questionnaire 
was too long, or questionnaire items required more ef f o: t than 
respondents were willing to give. For example, one respondent 
referred to the series of income questions as "a mini tax return." 

Those interviewed were asked if they would have been more 
inclined to participate if they had had more information about the 
study. Most were not interested in additional information. 

Asked if they would have been willing to participate in a 
telephone or in-p^rson interview, about two thirds of the H3&B 
"nonrespondents" indicated they would have been willing to 
cooperate in one of the alternative methods. About half of the 
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NLS-72 survey members said they would. Of course, these responses 
have to be considered with appropriate reservations since those 
contacted were asked about a hypothetical situation — they were not 
asked to actually complete the questionnaire. 

Survey members were asked if a larger incentive would have 
encouraged them to participate, and almost all indicated it would 
not have made a difference. 

These postfield interviews offer additional evidence that 
there is a significant respondent burden imposed by the current 
length of the questionnaires. That burden must be reduced if 
desired completion rates are to be reached during the full-scale 
survey . 

1.3.3 Data Preparation 

Questionnaires returned by respondents were reviewed by the 
NORC coding staff and edited for readability and completeness. 
Coders supplied codes to indicate if respondents made errors (such 
as failing to follow instructions) or if questions were skipped 
inappropriately. Coding specifications were written to preserve 
respondent error and missing data. There was no validation or 
retrieval of data from respondents. 

Comments that respondents had written in questionnaires were 
noted during the coding/editing process. Respondent comments 
together with coder evaluations provide the bases for the 
qualitative assessments of questionnaire items throughout this 
r .port . 

All questionnaire data were entered using a conventional key- 
to-disk method with 100 percent verification of data entry. Raw, 
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unweighted data were used for the system files created for 
analysis of fi'jld test data. Machine editing programs were not 
used. 
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2. NONRESPONSE AND RESPONDENT BURDEN 
2.1 NLS-72 

The NLS-72 fifth follow-up questionnaire was reviewed for 
item nonresponse. Sine** nonresponse may signal problems of 
clarity in question wording or instructions, it can serve as a 
diagnostic tool for determining revisions that need to be made for 
the full-scale survey. All items for which there was 4- percent or 
more missing data were examined. While many non-critical items in 
the full-scale survey have this much or more missing data, it was 
felt that setting the threshold relatively low would permit a 
closer examination of possible sources of confusion. The reader 
should note that missing data in this section will refer to 
illegitimate skips, that is, to questions that apply to all 
respondents or to a subgoup of respondents but which are not 
answered. 

All items with 4 percent or sere missing data are listed 
below along with a description of any problems and the proposed 
changes for the full-scale survey instrument. 

Item 4-: Did the respondent own a house, apartment, co-op, 
condominium or mobile home? 

Seven percent of the field test sample members did not answer 
this item. The item appeared at the bottom of page 1 , and 
therefore some respondents may have missed it. Simply placing the 
item in a more prominent spot may be enough to boost the response 
level . 



34 



Items 6F, 7F, 8F, 9F: Whether job was seasonal 

This item has not been part of the work history sections of 
the ELS questionnaires before. The missing data level was 4 
percent to 6 percent. Comments on some questionnaires indicated 
respondents were unclear about the definitions of "seasonal." A 
couple of examples will be added to the main survey instrument. 

Items 6G, 6H, 7G, 7H, 8G, 8H, 9G, and 9H: Job salaries 

These axe items within the job history sections that ask 
respondents to indicate their starring and ending or current 
salaries for each job they indicate. The level of missing data 
ranged from 4 percent to 7 percent, and this is somewhat lower 
than the levels of missing data that occurred for these same items 
in the second follow-up survey. The experience of the ELS surveys 
has been that respondents have difficulty providing complete and 
accurate information on dollar amounts, ?jid this is just another 
example of the problem. Since the level of missing data is ? wer 
than that of the HSftB second follow-up survey , no further changes 
are recommended in question wording. 

Item 11 and subparts A, B, and C: Employer-provided training, 
number of hours per veek and number of weeks 

Missing data on this item and the subparts ranged from 10 

percent to 20 percent. Many respondents answered the initial 

filter item here but then failed to complete parts A, and C. 

Also, those respondents who had received some training benefits 

often indicated the number of hours per week and the number of 

weeks they had spent in various kinds of training without 

indicating they had received such training. For a full 
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discussion, ate ''New Questionnaire Items— -NLS-72 Core 
Questionnaire. " 

Item 12: Was training aimed at retraining respondent because job 
was ph^ed out? 

This item was not answered by 19 percent of the respondents. 
Again, see the section on "New Questionnaire Ixems — NLS-72 Cere 
Questionnaire" for a full description. 

Items 13A and 13B: Whether the respondent has had any training 
since 1979 to improve skills within current job and/or to 
prepare for a new job? 

These two items were not answered by 7 percent and 9 percent 
respectively of the respondents. See "New Questionnaire Items 
NLS — 72 Core Questionnaire." 

Items 23 through 26: Education costs and forms of financial aid 
received for schools attended since 1979 

This section begins with an item asking respondents to 

indicate the two most recent time periods they were in school. A 

sequence of questions then follows about the cost of tuition ajid 

fees, room and board at these schools, and loans and grants and 

scholarships received during those time periods. Before each 

subpart, the respondent must repeat the two time periods he or she 

is describing. This was done in order to ensure that the 

respondent kept the **ame time periods in mind as he or she 

answered the sequence of questions. According to Q.23, 151 of the 

506 respondents had obtained schooling since 1979. 

Table 2.1-1 shows the high level of item nonrespons* for this 
section. For instance, in Question 23 respondents are asked first 
to label two columns according to the time period(s) they had 
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attended school- Then they are instructed to enter separate 
dollar amounts for tuition, room and board, and total costs in the 
colummns that correspond to the two time periods- They must not 
only remember the information called for but also enter it in the 
right place. 

Editors noted that errors were made frequently on these 
items — amounting in some cases to 4- percent of the respondents. 
The extent of nonresponse and error for these items suggest that 
respondents found it difficult to keep time periods and 
corresponding costs (and resources) clearly in mind, and/or had 
difficulty entering them in the appropriate places. 

Table 2.1-1 

Percent of Item Nonresponse for Sc ol Finances Q nions 



1st Period ! 2nd Period 

I 





! Eligible 


All ! 


Eligible 


All 


Question 


S Respondents 


Respondents ! 


Respondents 


Respondents 



Question 23 



Year 8 23 

Tuition & Fees 25 

Room A Board 30 
Total Annual Costs 29 

Question 24- 

Years 65 

Years (A) 72 

Years (B) 61 

Question 25 

Years 50 

Years (A) 57 



11 41 18 

12 42 19 
14 45 20 
14 45 23 



11 65 10 

11 75 11 

9 65 10 



12 58 12 

14 59 12 
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An additional problem of this section was that very few 
respondents could identify specific kind3 of loans or financial 
assistance- For inatance in Question 24, only 26 persons in the 
first period and 28 persons in the second period indicated they 
had received a loan. About 45 to 50 persons did not provide 
answers- Going on to the types of loans received, Table 2.1-2 
displays the frequencies and missing data (legitimate skips) for 
specific types of loans for each time. period- 
Table 2-1-2 

Frequencies and Missing Data for NLS-72 Question 24A 



I First Period I Second Period 



Type of Loan 



N Missing 



N Missing 



Nat'l Dir Stud 4 

Fed GSL 15 

Nursing Loan 0 

State Loan 8 

Coll-Univ Loan 1 

Bank Loan 1 

Loan-Parents 6 

Loan Unknown 2 

Fed Insured Loan 2 

Other Loan 4 



43 3 51 

43 9 51 

43 2 51 

43 5 51 

43 2 51 

43 1 51 

43 6 51 

43 0 51 

43 2 51 

43 7 51 



Two points are evident from this table. First, missing data 
continues at a very high rate. About one-third of the 151 
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respondents who were eligible to answer did not supply 
information. Second, there is a very strong flannelling process 
going on: because most respondents had received their schooling 
by 1979, xhese question! applied only to a minority. Within this 
group a much smaller subset, 26 to 28 persons, received an/ loans, 
and within this subgroup only a handful could identify a specific 
kind loan for each period. All of this demonstrates that the 
school finance section applies to a minority of cases and in the 
end yields only very small frequencies. For the NLS-72 cases, 
receipt of school loans is a relatively rare event and therefore 
should not occupy much space in the questionnaire. 

Item 24-B asks respondents to enter the total dollar value for 
the loans received. Similar to Item 24-A, only 39 and 35 
individuals provided answers, and answers for 4-4- and 4-6 cases were 
missing. 

The next question (Q.25) asked respondents to indicate 
whether they had received resources ether than loans, such as 
grants, scholarships, or parental assistance, for any schooling 
they had received since 1979- For the sake of brevity, we have 
not included a table. However, the pattern was similar to that 
for the loan section. Sixty-three persons in the first period and 
4-4- persons in the second period indicated receiving aid, and the 
number of illegitimate skips ranged from 50 to 60. Again, very 
few respondents could identify specific kinds of assistance. The 
highest frequency for any particular type of assistance was 30, 
for use of savings or earnings during the first period of 
schooling. 
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Item 26 asked respondents about whether repayment had begun 
for any education loans received since leaving high school. Data 
was missing for about 4-5 cases, or 9 percent of the total group. 

The relatively high level of missing data in this section 
suggests that respondents had some problems following the sequence 
or that this information is difficult for respondents to remember. 
For the field test, they were asked to review six years of 
activities, and for the full-scale survey, they will have to 
recall events over a 7-year period. Furthermore, even if 
respondents can recall the dollar amouncs, types of loans and 
assistance, relatively few of them reported receiving loans and 
assistance from government programs or other sources. 

In order to obtain more information and reduce the amount of 
missing data, we would suggest that the section on postsecondary 
education costs be consolidated. Instead of asking respondents to 
identify ahead of time two time periods they were in school and 
then target their answers to the two time periods, we recommend 
asking them to provide information on loans and assistance 
received since 1979. Simply stated, the question sequence might 
read as follows: 

Did you attend school between 1979 and 1986? Yes No (skip) 

Did you receive any loans? Yes No (skip) 

Types Number Dollar 
of loans........ of Years Amount 

Did you receive any other kinds of assistance, such as 
scholarships, grants, parental assistance, etc.? 

Yes No (skip) 

Kinds of Number Dollar 
Assistance of Years Amount 
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Such an approach would simplify the sequence, reduce the 
onerous instructions to the respondent, and eliminate the need for 
him or her to constantly record th** same periods of time. Also, 
since the respondent would not be limited to two time periods, he 
or she might have more to report, thus increasing the chances that 
the use of various typ*s of assistance would be captured more 
completely. 

Items 27, 28, 29: Days spent with parents, financial assistance 
from parents, and whether respondent is claimed a* a 
deduction 

The level of missing data for these items was 7 to 10 
percent. This compares to 3 to 6 percent for comparable items in 
the the senior c ort questionnaire used in the HS&B second 
follow-up survey. Since these respondents axe supplying 
information for six rather than two years, this question may 
appear burdensome and discourrf.ge pp pie from answering. Results 
showed that from 375 to 4-1 4- respondents that indicated for «he 
years given they had not lived a single day with their pa:ents. 
Therefore, we recommend that a response alternative be added that 
allows the respondent to check that he or she spent 0 days living 
with his or her parents during the years listed. The addition of 
an alternative covering the entire time period 3hould reduce the 
burden for participants. 

The same type of alternative should be created for the 
question on financial assistance (Q.28). Respondents should be 
able to check that no financial aid was received during any of the 
years listed. 
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Item 89: Number of children respondent has ever had 

Thirteen percent of the respondents did not supply an answer 
to this question. This item is embedded in a skip pattern 
beginning at item 87, which asks the number of children 
respondents expect to have. The coding staff noticed that the 
skip instruction, which allows respondents to skip items on 
children if they do not anticipte having children, wac placed in 
an ambiguous position. Forty-three respondents indicated they 
expected to hav* a certain number of children but then failed to 
answer the item on children. 

Ve recommend that item 87 be revised to ask respondents to 
write in the number of children they expect. A more complete 

istruction should be given for "0": "If you expect '0» children, 
skip to Q, 106." Making the skip instruction more explicit should 
reduce confusion and the level of missing data. 

Item 119: Social Security Ni:*nber 

The missing data on this item was 5 percent. At this time, 
we have no recommendations to make regarding any wording or format 
changes. Telephone retrieval may represent the best way to boost 
response rate. 

Item 122: Income 

Respondents were asked to supply information on fourteen 
different sources of income they may have received over the past 
two years. Income sources are listed in two columns, one for each 
year, and the respondent is supposed to add them up and enter the 
total for each column. If the respondent has no income from a 
particular source he or she is instructed to enter "0", When the 
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respondent did not enter anything, the item was classified as 
illegitimate skip. For 1984, the missing data ranged from 67 to 
113 cases or 13 percent to 22 percent. For 1985, it < as somewhat 
higher: 100-136 or 20-27 percent. This compares to about 3 
percent for the HSftB second follow-up senior cohort. 

There are two explanations for the differences in missing 
data. First, there was an error in the years listed on the field 
test. Instead of 1984 and 1985, the years should have been 1983 
and 1984-, Since it was impossible to provide complete information 
on 1985, respondents may have been unclear about what they were 
supposed to do. The second and more important explanation for 
differences between the NLS-72 field test and HS&B income items is 
that income was ritical item on HSAB. When the item was left 
blank, respondents were telephoned and asked to supply missing 
information. Should income be declared a critical item for the 
fifth follow-up survey, it appears that considerable effort will 
be reeded to obtain complete information. 

The fact that the income item is located near the end of a 
very lengthy questionnaire suggests that respondents become too 
tired and unwilling to provide complete answers. One, respondent 
wrote in a comment that said this was too tedious and likened it 
to a "mini income tax form," Shortening the overall questionnaire 
may reduce fatigue enough to increase the response rata to this 
item. 

2.1,2 Implications of Nonresponse for Re;rieval 

While mailed questionnaires are much less costly than 
personal interviews, they a?e more susceptible to missing data, 
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since an interviewer is not present to see that each qaestion is 
answered. In order to alleviate this problem, "critical" items 
will be identified which will be subject to retrieval by 
telephone- These items are so designated if they are deemed 
crucial to the methodological or analytical objectives of the 
study. Most of the key items are of self-evident policy 
relevance; others are usually chosen as a means of checking 
whether survey respondents properly followed routing instructions, 
or whether they inadvertently skipped portions of the 
questionnaires. 

During the main survey, when a questionnaire arrives, it 
passes through a Critical Item Edit process which flags any 
critical item that has not been answered. The questionnaire is 
then routed to a data retrieval station where respondents are 
called by telephone to obtain missing information and r.lear up any 
ambiguities in their responses. During the HS&B secc- i follow-up 
survey, about 29 percent of the sophomore questionnaires and 
33 percent of the senior questionnaires required retrieval. 

As we prepare for the full-scale durv*y, it is important to 
assess the level of nonresponse for items' which axe prime 
candidates for retrieval. This assenment helps us judge the 
magnitude of the task, the number of persons needed to carry out 
retrieval, and the costs involved. Table 2.1.1-1 below lists 
possible retrieval items and the percent of missing data that 
occurred in the field test. The items selected represent items 
that have been designated critical in past HS&B surveys and items 
which are filter questions. 
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Table 2,1 .1-1 
Nonresponse Rates for Potential Critical Items 

NLS-72 



Itei 


Content 


Percent 
Illegitiaate 

Skip 


1 I tea 


vUIl V6II w 


Percent 
Illegltlnate 

dlVlp 


T 

J 


What were you doing the first 


1 


1 8E 


When were you working at this 


3 




week of Feb 1985 






Job 




1U7 


With whoa did you live the first 


3 


1 14 


Between Nov 1979 & Feb 1985. 


3 




week of Feb t 985 






were vou un»anl nv»H 




ft 1 


now far in scnooi ao you 


6 


1 14B 


Did you receive uneaployaent 


2 




think you will get 






insurance 




15 


Between Oct 1979 t Feb 1985, 


1 


1 82 


Marital status as of Feb 1985 


6 




did you attend any school... 
















1 36 


How aany tlaes were you aarried 


2 


17 


First postsecondary school 


1 






17B 






\ 89 


How aany children have you ever 


15 


When did you attend (calendar) 


2 




had 




18 


Second postsecondary school 


1 


98a 


What were chlldcare arrangeaents 


P 


18B 








Tt*T nrpvchnnlprv 




When did you attend (calendar) 


1 






25-1 






98B 


What were chlldcare arrangeaents 


19 


Total tuition and fees 


12-19 




for school-age children 




23-3 


Total cost of schools 


14-20 


106 


During spring 1972, with whoa were 


4 










you living 




248 


Total value of loans 


9 






26 






117 


What is your origin 


2 


Have you begun repayaent of 


9 








of education loans 




116 


What is your race 


1 


5 


Between Oct 1979 I Feb 1985 


1 


122 


Incoae 


13-27 




did you ho?d full-tiae or 












part-tiae job 




125 


■4ave vou ever cansidered 


ft 




pi — _X J - L ,1.-1 | J * _A J . t_ 

First Job — what k:nd of Job 






eleaentary, secondary teaching 




0 


1 






6E 


When were you working at this 




124 


If conditions were right, would 


4 


2 : 




you consider going Into eleaentary 






Job 






or secondary teaching 




7 


Second Job— w?mt kind of Job 


2 ! 


Tsr 


Which of these groups defines you 


7 


7E 


When were ycu Norking at this 


2 


TS3 


In which fields vera you 


2 




Job 






certified 




8 


Third Job— what kind of job 


3 


TS5 


Did you teach during the 


0 










1984-85 school year? 





Note: Percentages represent the number of illegitimate skips divided by 506. 

For the three Teaching Supplement items, the number missing is divided 
by 126. 
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Most of these items have relatively low nonresponse rates. 
However, these percentages may underestimate the actual level of 
missing data to expect in the main survey. It should be 
remembered that the field test has a response rate about 10 
percentage points below the main survey, and this last 10 percent 
often represents the respondents who have to be prompted more 
often and generally axe less cooperative with survey 
participation. Their questionnaires may very well be less 
complete. 

By far the highest level of nonresponse occurs for the school 
financing questions, the number of children, childcare 
arrangements, and income. All of these items have been discussed 
earlier in this section aio::g with recommended actions for 
instrument modification. Some reduction in nonresponse should 
result from questionnaire revisions. Yet we can probably still 
expect moderate levels of nonresponse for these items, pe -haps as 
much as 10 to 1 2 percent. 

2.1.2 Respondent Burden 

The final question answered by the NLS-72 field test sample 
members was, "Altogether about how many minutes did it take you 
to complete this questionnaire? 11 The results are displayed in 
Table 2. 4 .2-1 . 
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Table 2.1 .2-1 

Minutes Required to Complete the NLS-72 Fifth Follow-Up 
Field Test Questionnaire 



Cum 



Minutes 


N 


Percent 


Percent 


15-30 


63 


13 


13 


31-45 


98 


20 


33 


46-60 


122 


25 


58 


61-75 


56 


12 


70 


76-90 


77 


16 


86 


91-105 


5 


1 


87 


106-120 


40 


8 


94 


over 120 


28 


6 


100 



The survey design called for a questionnaire which could be 
completed in about 45 minutes hy most respondents, but the mean 
number of minutes required was 69. Table 2.1.2-1 shows that only 
twenty percent of the respondents could complete the questionnaire 
in 4-5 minutes or less. A little over half the respondents were 
able to complete the survey in an hour or less time. However, 
4-2 percent required more than an hour, and 6 percent required more 
than two hours. 

Clearly, with two-thirds of the sample requiring more than 4-5 
minutes *o complete the survey, the questionnaire is longer than 
was intended. This is not surprising because the field test 
usually is longer than the full survey in ordur to permit the 
evaluation of a longer set of questions. Field test results are 
then used to determine what items will be kept and which items 
will be eliminated in the final document. 

Because supplementary studies that add more items to the 
questionnaire have been approved since the field test began, it 
will be especially important to look for items to eliminate. The 
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Spencer Foundation supplement alone will add approximately four 
pages of attitude questions to the existing NLS-72 core 
questionnaire. Also, a few more questions will be added to the 
core questionnaire and the Teaching Supplement as part of the 
additional National Science Foundation grant. Furthermore, the 
2,4-00 persons who were drawn into the sample to complete the 
Teaching Supplement must complete both the core questionnaire and 
the supplement. These persons will receive a higher incentive 
payment, but we cannot be sure that it will offset the fairly 
heavy burden of completing both questionnaires. Ve know from past 
experience that as burden increases, willingness to participate in 
the study decreases. This point was reinforced by the telet)hone 
interviews we conducted with a group of field test nonrespondents- 
Most of them cited the length of the quests *aire and lack of 
time as the main reasons for nonparticipation. Hence, a 
burdensome questionnaire could threaten the goal of attaining a 
completion rate of 90 percent. In addition, it would elevate the 
cost of data collection and preparation. 

3.2 High School and Beyond 

The High School and Beyond field test questionnaire was 
examined using the same criteria as the NLS-72. Each item with 
4- percent or more illegitimate skips was examined to determine if 
some improvement in instructions, question wording or sequence 
could be made. 
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Items 8G, 8H, 9G, 9H, 1<?0, 10H, 1 1 G, 11H: Starting salaries 
and wage types and ending salaries and wage types 

These item? ask the beginning salary and wage type for the 

four jobs that respondents can report. Illegitimate skips ranged 

from 2 to 6 percent. This level of missing data is comparable to 

that obtained in the second follow-up of High School and Beyond. 

No recommendations for change are indicated. 

Item 12: Did respondent have full-time job between March 1983 
and February 1985 

This question, which was not answer. d by 5 percent of the 
respondents, is a filter item for the sequence of questions having 
to do with on-the-job training. The items are identical to the 
NLS 72 series beginning rith Q.11. The suggestion for that item 
applies here as well. The question and its subparts should be 
laid out to make the second skip opportunity more explicit. See 
the NLS-72 section on new questionnaire items for a full 
explanation. 

Item 13: Received formal training outside the job 

Five percent of the respondents did not answer this question. 
This compares to *bout 4- percent during the second follow-up 
survey for the same item. Editors noted that under 13A where 
respondents could write in the place where training was received, 
they often wrote in "on-the-job." This indicates that respondents 
are somewhat confused about what is desired in this answer. We 
recommend that the instruction be revised to emphasize that they 
should include only formal training programs and not on-the-job 
training. Further clarifying the question may improve the 
response rate. 
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Items 13E1, 13E2, 13E3, 13E4: Availability of equipment used in 
training, whether it was in good repair, up-to-date, and 
provided preparation for job 

About 4 percent of all respondents and about 10 percent of 
those eligible to answer this series of questions did not do so. 
This level of missing data is still below that encountered in the 
second follow-up survey for this same set of questions, and 
therefore no changes axe recommended at this time. 

Items 35A through 35J: How many years of coursework taken in ten 
different subjects 

The missing data ranged from 3 to 7 percent on this set of 
items . On the comparable items for the second follow-up 3 to 
4 percent of the senior cohort and 3 to 7 percent of the sophomore 
cohort did not answer. Hence the level of missing data is the 
same, and no recommendations for change are made at this time. 

Item 36: Did the respondent attend school during 1983-84 and 
during 1984-85 

Four and 5 percent of the field test participants 
respectively did not indicate whether they had attended school 
during these years. Missing data for this series in the second 
follow-up was less than 1 percent, and this is explained by the 
fact that it was a critical item. The level of missing data in 
the field test suggests that the item should remain on the 
critical item list. 

Items 36A, 36B, and 36C: Tuition, room and board, and total cost 
of schooling for 1983-84 and 1984-85 

Respondents were to enter dollar amountb for tuition, room 
and board, and the total cost of schooling for each year. Missing 



17 



ERIC 




data ranged from 4 to 6 percent for these items* During the 
second follow-up survey, questions on tuition and fees and total 
costs were critical items, and missing data were held to 1 
percent. Missing data for room and board values, which was not a 
critical item, was about 5 percent. Therefore, the level of 
missing data for the cost of schooling appears to be comparable to 
that for the second follow-up survey, and the response rate will 
need to be improved through telephone retrieval. 

Item 37, 37A and 37B: Whether the respondent received 

scholarships and grants, kinds of scholarships and grants, 
and dollar amounts for 1983-84- and 1984-85 

From 4- to 5 percent of respondents did not provide 

information in thi3 section. This compa. _s to ,5 to 1 percent for 

the second follow-up survey (both cohorts). Considering the size 

of the sampling error for the field test, which is a little over 2 

percent for these items, nonresponse does not appear to differ 

appreciably from the second foilow-up. No changes are recommended 

at this time for these items. 

Item 39: Whether respondent received financial assistance from 

parents, spouse, other relatives; amount of aid from parents; 
and amount of aid from spouse, other relatives 

On the first part of this item, the level of missing data 

was 5 percent. For the dollar amounts, parental aid was not 

indicated by 5 percent for 1983-84 and 7 percent for 1 984-85, 

About 15 percent of respondents failed to specify uhe amount of 

assistance received from spouse, friends, relatives. Missing 

data on comparable items from the second follow-up was 4 to 

5 percent. 
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A close inspection of this item revealed that one instruction 
that had been part of the question in the second follow-up 
questionnaire was inadvertently omitted from the section on dollar 
amounts. This instruction asks respondents to enter a "0" if they 
received no financial assistance. Without entering the "0", a 
response that is left blank remains ambiguous. It is impossible 
to know whether no assistance was received or whether the response 
is an illegitimate skip. For the full-scale survey, the 
instruction will be added. 

Item 4-0A and 4-0B: Sources of money paid by the respondent for his 
or her schooling and dollar amounts 

Missing data amounted to 5 percent of these two items and 
subsections. This compares to 3 to 5 percent on the comparable 
item in the second follow-up survey (both cohorts). Since the 
level of missing data is similar to that for the second follow-up, 
no changes are recommended for this item. 

Item 43C: Whether respondent was claimed as a tax exemption 



Respondents may have beer unsure of this item because the 
missing data jumped from 3 percent for 1983 and 1984- to 4 percent 
for 1985. Since the questionnaire was mailed out in early 1985, 
respondents may not have known yet whether their parents would 
claim them as an exemption. For the second follow-up, which 
occurred in 1 984 , respondents were asked only about prior yearn 
(up to 1983), and the missing data level was comparable to this 
field test. We re ;ommend that the present year, which will be 
1986 for the main survey, be eliminated for this question. 



in 1985 
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Item 45: Whether five different needs were provided to the 
respondent by his or her parents during 1983 and 1 984 

Participants were asked to indicate whether room, board, the 

use of a car or truck, medical expenses, and clothing were 

provided by their parents in the years listed above. About 6 

percent of the respondents skipped this item illegitimately. This 

is comparable to the second follow-up survey where the missing 

data was 4- to 6 percent. No changes are recommended for this item. 

Item 4-9: Whether the respondent has participated in non credit 
courses, correspondence courses, courses given by community 
group, and so on 

Six percent of the respondents skipped these questions. 
During the second follow-up survey this was a critical item, and 
therefore the missing data was less than 1 percent. No changes 
are recommended for the item. To achieve a comparable response 
rate in the third follow-up survey, it will be necessary to keep 
this item on the critical item list. 

Item 66: Income 

For this item respondents were asked to indicate for 1983 snd 
1984 how much income they had received from 13 sources as well as 
the total amounts of income from all sources. They were 
instructed to write in M 0" for all sources from which they did not 
receive income. Regardless of the instruction to enter a value or 
a '0 M in each space, 34 to 54 individuals, or 10-16 percent, left 
items blank on this question. During the second follow-up survey, 
this ^as a critical item, and the missing data only amounted to 
about 3 percent. No change in format is recommended. However, 
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these results suggest that, to achieve high item response, the 
question should remain on the critical item list. 

Item 77: Whether respondent has any specified disabilities or 
handicaps 

Respondents were to indicate whether they have a learning 
disability, visual handicap, difficulty hearing, deafness, 
orthopedic handicap, or other physical disbility. The nonresponse 
rate for this item was 9 percent. It is worth noting that the 
question applied to relatively few persons, with frequencies 
ranging from 0 on deafness to 10 for "hard of hearing." Given the 
high nonresponse rate and the fact that these conditions are rare 
events, we recommend that this question be considered for 
elimination. 

# 

2.2*1 Implications of Nonresponse for Retrieval 

r Jable 2,2, 1-1 lists the items which are likely to be critical 
and require telephone retrieval • 
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Table 2.2.1-1 
Nonresponse Rates for Potential Critical Items 
High School and Beyond 



Item Content 



Percent 
Illegitimate 
Skip 



3 What were doing the first 0 
week of Feb 1985 

4 Uith vhor. did you live the 2 
first week of Feb 1985 

7 Between Harch 1983 and Feb 193F did 3 
you have a full time or part time job 

8A /irst job— vha ; kind 2 

8E When were you working at this job 3 

9A Second job— what kind 2 

9E Uhen were you working at this job 2 

10A Third job— what kino 4 

10E Uhsn were yo*j working at this job 3 

21 Job you expect to ^.se at age 30 1 

22 between Harch 1983 and Feb 1985 1 
were you unemployed 

23 Did you complete high school .3 

25A Uhen did you earn the high school .3 
diploma 

24 How frr in school do you think you » 
will get 

25 Sincr high school did you take .6 
classes Tor credit 

30 Between Harch 1983 and Feb 196* did 1 
you attend or take classes at a 
college or university 

31 First P <E scl ool 2 
31C Mhen did you attend this school 3 

32 Second r£E school 3 
52C Uhen did you attend this school 3 



Itea Content 



Percert 
Illegitimate 
Skip 



36A Total annual charges for tuition 5 
for each year attended 

SbC ,„cal annual cost cf attending 6 
school each year 

37B Totail dollar value or scholarship 5 
or grant for eaiA year 

38B Total dollar value of loans fw 4 
each year 

42 Since leavi , high schoc' have 2 
you received ;ducational loans 

42A Any student loans oeferred or 3 
forgiven 

42B Has repayment of education 2 
loans begun 

43 Here you claimed by parents as 3 
m exemption in 1983 and 1984 

47 Since Harch 1983, have you 2 
applied for graduate or 

professional school 

47B Did yov apply for financial aid 2 

48 Between Harch 1983 and Feb 1985 t have 1 
you participated in an apprenticeship 

or manpower training program 

49 Between Harch 1983 and Feb 1935, have 5 
you taken non-cred<*. courses, 
correspondence cr ses, etc. 

50 Harital status as of Feo IVo? .5 

55 How many times haved you been .5 
married 

58 As of Feb 1985 , how many children .5 
66 income 15 



Note: Percentages represent the ^unber of illegitimate skips divided by 334-. 
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Unlike the NLS-72 survey, there is only one item with 
relatively high missing data, and this is income. The other 
potential critical items range from 0 to 5 percent in the level of 
missing data. Considering these estimates a3 well as the 
similarity of the third follow-up survey to the second follow-up, 
we expect that ths amount of telephone retrieval will also be 
comparable. As mentioned earlier, during the second follow-up 
survey, 29 to 33 percent of the questionnaires required some 
telephone retrieval. 

2.2.2 Respondent Burden 

At the end of the High School and Beyond questionnaire, 
respondents were asked to indicate the number of minutes required 
to complete the questionnaire. The mean number of minutes was 68, 
and Table 2.2.2-1 displays the distribution of participants across 
time segments. Sixty-six percent of the respondents were able to 
complete the questionnaire in an hour or less, ana 44 percent 
required more than an hour. Two hours was required by 6 percent. 



Table 2.2.2-1 

Minutes Required to Complete the HS4B Third Follow-up 
Field Test Questionnaire 



Minutes 



N 



Percent 



Cum 
Percent 



15-30 
31-45 
4-6-60 
61-75 
76-90 
91-105 
106-120 



46 
62 
107 
25 
37 
7 
20 
19 



14 
19 
33 
8 
1 1 
2 
7 
6 



14 
33 
66 
74 
85 
87 
94 
100 



over 120 
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While these results appear encouraging, they are somewhat biased 
towards those individuals who take less time. As explained for 
the NLS-72 sample, these results reflect an overall response rate 
of about 80 percent. For the main survey, we will be pursuing 
another 10 percent to reach the 90 percent response rate. Past 
experience indicates that this latter group often needs assistance 
with the completion of the instrument through a telephone or 
personal interview. In many cases, their reading skills are not 
as good, and therefore they are likely to require more time to 
complete the questionnaire. 

It should be remembered that the field tPSt instruments were 
purposely designed to be longer than the instruments for the 
full-scale survey. They are allowed to become more lengthy in 
order to fully test new items to determine which should remain in 
the survey ai x which should be eliminated. In order to keep the 
questionnaire at a reasonable length, it is recommended th-=it no 
further items be added, and that consideration be given to 
eliminating any items that do not add appreciably to the results. 
In order to maximize the chances of reaching the desired response 
rate while staying within budgeted costs, it is important to keep 
the respondent burden to no more than one hour. 
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3. LOGICAL CONSISTENCY OF RESPONSES TO FILTER AND DEPENDENT 
QUESTIONS 

Among the most effective methods of minimizing survey response 
burden is the use of "skip" or "routing" instructions to direct 
subjects to answer only those items relevant to their experiences. 
"Filter" questions are used to ascertain whether or not 
respondents have had specified experiences (e.g., postsecondary 
schooling, work outside the home, marriage, having children). If 
the response is positive, subjects axe directed to a set of 
follow-up or "dependent" questions that ask for details about 
the experiences. If the response to the filter item is negative, 
the subject is directed to skip over the dependent terns focusing 
on that experience, and to resume the questionnaire at a 
designated "target" question. For example, in the case of 
postsecondary schooling, respondents who report having attended 
during a specified time period are asked a series of fol]ow-up 
items about the type of school attended, their level of 
involvement in course work, their field of study, and so on. 
Those who report no schooling beyond hign school are directed to 
skip to a target item dealing with a new topic. 



The use of skip patterns in questionnaires improves the 
efficiency of the data collection experience by eliminating the 
need for rccpondents to indicate that specific survey items are 
"Not Applicable" to them. At the same time, this device impose j 
some risk that respondents will- fail to observe properly the 
routing instructions, introducing various types of response error. 
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First , respondents may fail to mark an answer to the filter item 
so that it is incertain whether or not the subject has had the 
experience in question. This is potentially the most serious 
error , because it clouds the specification of the population who 
should be answering the sometimes lengthy series of follow-up 
questions. Second, respondents may fail to understand the 
instructions accompanying the filter item, and may omit the 
dependent items when they should have answered them. Third, 
respondents may answer the filter negatively, but still attempt to 
answer some or all of the follow-up items when they should have 
skipped them* Finally, respondents answering the filter 
negatively may properly omit the dependent items, but fail to skip 
to the designated target question, inadvertently missing one or 
more items. 

These response errors decrease the potential efficiency of 
the routing patterns because they require either callbacks to 
respondents who fail critical items edits (most major filter 
questions are designated critical items;, or extensive programming 
to computer-edit the responses into consistent r*tterns. Both of 
these p.ctivttics are costly and should be minimized. Fail-edit 
callbacks require brief but properly supervised reinterviews and 
thus increase response burden. Machine (computer) editing 
requires extensive planning and checking, and extendi* the time for 
delivery of public release tapes. 

Thus, one of the most important activities in questionnaire 
design and revision is the development of skip patterns that will 
minimize the several types of response error. If the risk of 
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response error is excessive, for example in surveys of younger 
children or poor readers, or if the costs of correcting the errors 
are unacceptable, skip patterns can be eliminated. For example, 
because of cost constraints on the amount of editing possible for 
the very large base year samples, skip patterns were almost 
coropl> ely eliminated from the HSftB base year questionnaires. 
Respondents were expected to answer every item in the 
questionnaire, even if the response was "Not Applicable." To 
reduce the time required to complete the survey, skip patterns 
were employed in the first and second follow-up questionnaires, 
with excellent results. 

The HS4B third follow-up and NLS-72 fifth follow-up 
field test questionnaires include a large number of skip pattern, 
in order to maximize the amount of detailed data obtained from 
respondents who have had experiences of direct relevance to 
educational policy (especially postsecondary schooling and 
financial aid). In several instances, minor skip patterns are 
nested within larger skip patterns, increasing the complexity and 
the possibility of response error. For example, HSftB Q.50 asks the 
respondent's marital status as of the first week of February 1985. 
Those not currently married (or living xn a marriage-like 
relationship) are directe ' to skip to Q.57, while those who are 
married or living with someone of the opposite sex are directed to 
answer the immediately following items. Within the follow-up 
series, question 52 asks whether the respondents "spouse" held a 
job during the first week of February 1985. If the answer is 
"yes," the respondent is asked a series of 7 items in Q.53 about 
the spouse's job. If the answer is "no," the respondent is told 
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to skip to Q.54~-still within the original set of dependent 
questions related to Q.50. 

In a very few instances, skip patterns are nested twice. 
That is, tertiary skip patterns are placed within secondary skip 
patterns that occur wi*nin primary skip patterns. Usually these 
patterns involve only one potential dependent item. 

The remainder of this section reviews the empirical results 
with respect to response errors to skip patterns in the field test 
questionnaires. The appendix to this section contains tables 
showing the numbers of respondents answering each filter item and 
the numbe: s commiting each type of error. Since the numbers of 
response errors are generally too small to merit attention, the 
discussion here will be limited to an overview of the results for 
major skip patterns and an analysis of special problems requiring 
revision. 
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HS4B Third Follow-Up Field Test Questionnaire 

The HSAB third follow-up questionnaire included over 20 majnr 
skip patterns: 



Filter 


Target 


Topic 


► Q.7 


Q.20 


Work experience 


Q.8-1 


Q.12 


Second job held 


Q.9-1 


Q.12 


Third jrb held 


Q.10-1 


Q.12 


Fourth job held 


Q-12 


Q. 13 


Employer provided training 


Q- 13 


Q.H 


Formal training for job 


Q.22 


Q.23 


Periods unemployed 


Q.25 


Q.48 


Postsecondary schooling 






since high school 


Q.30 


Q.48 


Postsecondary schooling 






during last 2 years 


Q.31-j 


Q.33 


Second postsec. school 


Q.37 


Q.38 


Non-repayable financial aid 


Q.38 


Q.39 


Repayable financial aid 


Q.39 


Q.40 


Financial aid from parents, 






spouse, other relatives 


Q.40 


Q.41 


Use of own savings or earnings 






for educational costs 


Q.42 


Q.43 


Student loan repayment 


Q.47 


Q.48 


Application to graduate or 






professional school 


Q-48 


Q.49 


"Other" training 


Q.50 


Q.55 or 


Marital partner if ever married 




Q.57 




Q.52 


Q.54 


"Spouse' s" job 


Q.55 


Q.57 


Marital history 


Q.58 


Q.66 


Parenting history 



Items marked with an asterisk (*) are filter items asked of all 
334 field test respondents. Others are major secondary skip 
patterns placed within the primary patterns. Some minor skip 
patterns are omitted from the list above. 

In general , the field test encountered very few instances of 
respondent failure to answer the major filter questions. For the 
items listed above, on ^ *rage fever than 8 cases (2 percent of 
the field test sample of 334) failed to answer the filter item. 
Thus for an average of 98 percent of the respondents, it was clear 
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from their answer to the primary or secondary filter question 
whether they should be attempting to answer the specific dependent 
items for a topic area. The variability in error percentage 
around this average level was also low. In the most extreme case, 
20 respondents (6 percent) omitted a secondary filter item (Q.36- 
b) on the periods of postsecondary school attendance. This very 
low level of response error is in keeping with that found for the 
full-scale HSftB first (1982) and second (1984) follow-up surveys. 

The observed error rate for following "SKIP TO" directions to 
the designated target item was also very low. Only 2 or 3 
responden+s, on average, or about 1 percent of the HwAB field test 
sample failed to answei the target item associated with each 
filter question. 

Once the filter item was answered, respondents generally 
followed the instructions at the filter very accurately A 
sample of over 80 dependent questions related to the fillers 
listed above were examined for response errors (see the appendix 
for a specific list of dependent items checked). On average, 
fewer than 7 respondents (2 percent of the sample) failed to 
follow the filter item instructions and omitted responses to the 
related dependent questions. Also, on average, fewer than 2 
respondents supplied responses to the dependent items when their 
response to the filter indicated that they should have skipped 
these items. Thus, the average overall level of response 
inconsistency between filter and dependent items is so low that 
few revisions to the routing patterns in the questionnaire seem 
warranted. 
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A few specific cases deserve mention, however. These will be 
discussed by topic area below. In most cases, we believe that 
routing pattern problems in field test data may be corrected 
either by the addition of a filter question to clarify the meaning 
of the question and the associated instructions, or by 
improvements in the formatting and layout of the problematic 
pages. 

Employment Histories (HS&B) 

The routing patterns beginning with Q.7 and continuing 
through Q.12 posed almost no problems for respondents. Worth 
noting, however, is that 23 respondents omitted Q.8-h, and 11 
respondents omitted Q.9-h (ending or current salaries for first 
and second jobs reported). Although the level of nonresponse is 
relatively high, we do -not believe that this type of nonresponse 
is related to question wording or unclear instructions. Rather, 
it is typical of the slight elevation in nonresponse that 
invariably attends detailed questions about income and earnings. 
Often respondents cannot remember well the salaries they earned at 
previous jobs. Moreover, some respondents exercise their right to 
withhold a response to this voluntary survey. Ve therefore 
recommend no changes to this series of items. 

The questions on employer provided training (Q.s 12 
through 12-d), however, require clarification and some 
reformatting. Very few respondents omit the secondary filter 
Q. 12. Ve believe that respondents become confused about how to 
ar.swer parts 12-a through 12-c in part because the instructions 
are not sufficiently clear and in part because the response codec 
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adjacent to the 6 options comprising part 12-a are too difficult 
to see* As a result, there are more responses to parts 12-b and 
12-c than the number of respondents who report an experience in 
part 12-a, Furthermore, we believe that a substantial percentage 
of respondents who have had no employer provided training failed 
to circle the code "8" next to the first option under part 12-a, 
It thus appears that the amount of missing information for this 
series of items is very high at between 10 and 20 percent, 

Ve recommend revising this item to include a second filter 
question positioned directly after Q,12 (and prior to the 
existing part 12-a) that would ask whether or not respondents have 
had any t^pe of employer-provided training. The wording of the 
new item should include explicit references to the types of 
training that are relevant, and should match the wording used in 
the existing Q,12-a, Respondents who indicate they have had no 
training will be instructed to "SKIP TO Q,13," and those who 
report some type of employer-provided training will be directed to 
answer parts 12-a through 12-d, The option "No Training benefits 
or programs , , , 8" would be removed from the list under part 12-a, 
Finally, the instructions for answering parts 1?-a through 12-c 
would be clarified and the response option codes for part 12-a 
would be made more visible. 

One other revision is worth considering for the section on 
labor force participation. In the field test questionnaire, the 
designated target item for the primary filter question (Q,7) is 
Q.20, which asks about the importance of 10 factors for 
determining the type of work the respondent is likely to do during 
the rest of his or her life. The following item (Q.21) then asks 
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what type of job the respondent expects to have at age thirty. Ve 
believe that it would improve the logical flow of the 
questionnaire to switch the position of Q.s 20 and 21, so that 
the item on job expectacions at age thirty is asked first and i3 
the designated target item for Q.7, the emp3oyment filter. 

Periods of Unemployment (HS&B) 

Approximately one third of the respondents reported in 
answering Q.22 that they had experienced a period of unemployment 
between March 1983 and February 1985. Only 4 cases omitted the 
f/lter item. Very few (5 to 7) respondents were coded as missing 
on any of the dependent items. 

High School Completion (a minor skip pattern) 

The intention of Q.23 ("Did you complete high school?") is 
to obtain dat.i on respondents who may have earned high school 
equivalency certificates since the last follow-up. Apparently, the 
routing instructions associated with the response options are not 
sufficiently clear for respondents. Only 7 respondents indicated 
that they had not graduated on schedule, but had later earned a 
GED diploma. Only th™e respondents were directed to answer a 
follow-up item on the date they received the GED. However, 55 
respondents filled in a date at Q.23-a. 

Furthermore, only the 12 respondents who reported that they 
"Left without graduating" were directed to answer Q.23-b. Over 
twice that many provided an answer. These excessive responses may 
be reduced or eliminated by adding such phrases as "IF YOU EARNED 



9 



ERJC 




A GED DIPLOMA: H or "IF YOU LEFT WITHOUT GRADUATING: " to Q.s 23-a 
and 23-b. 

Postsecondary Education (HS&B) 

The first skip pattern vithin the section on postsecondary 
education begins at Q.25> which asks whether or not respondents 
have attended any school since leaving high sciirjl. Itens between 
Q.25 arid Q.30 have almost no response errors. Virtually all 
respondents who have had no schooling skip properly to Q.48. 

Question 30 begins a secc~.d major skip pattern. Dependent 
items in this series are to be answered only by respondents who 
have attended school during the last two years. About half of the 
HS&B field test sample reported attending at least one school 
between 1983 and 1985. Only 44 respondents (about 13 percent) 
attended more than one school during that period. With exceptions 
noted below, the rate of response error is also very low in this 
section. 

Modest levels of response inconsistency emerged for -he 
series cf items on financing postsecondary education. In general, 
between 5 percent and 5 percent of the field test sample (and 
between 6 percent and 10 percent of those who attended some school 
since March 1983) failed to answer dependent items between Q.36 
and Q.39 on school costs and various forms of financial aid. 
Missing data levels for Q.4-0 on use of respondents' savings or 
earnings for school costs were about half those for Q.36 - 39. 
f' dering the results to similar survey items that have bern 
asked since the first follow-up of NLS-72 in 1973, we conclude 
that the field test results are not due to problems peculiar to 
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the field test questionnaire. Historically , financial aid items 
have presented respondents with special difficulties in e ich of 
the full-scale surveys. The problems appear to be primarily those 
of lacking information on sv. 1 costs and aid awards, or of 
recall of specific dates and amounts of aid received, not question 
wording or format. Ve do not feel that any revisions to these 
items offer much promise of improvement in response consistency, 
but instead would be more likely to harm the comparability of 
responses over several survey waves. 

Very few respondents omitted *che items on parental borrowing, 
loan repayment, or the series of items related to 
dependent/independent student status. The largest amount of 
missing data (about 4- percent) was observed on Q.43-C, on whether 
the respondent would be claimed as dependents by parents on their 
1985 Federal income tax return. The reason for nonresponse in 
this case is clearly that respondents were not able to predict the 
answer. 

Family Formation (HS&B) 

The series on family formation behavior begins with Q.50 cn 
marital status as of February 1985. Only three respondents 
omitted this filter question. A total of 123 were directed to 
answer Q.s 51 through 54- as a result of their responses to Q.50. 
In general, respondents correctly followed the routing 
instructions in this section, properly answering all items on 
spouses and children. 
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NLS-72 Fifth Follow-up Field Test Questionnaire 



The NLS-72 field test questionnaire includes many of the same 
routing patterns discussed above for the HS&B questionnaire, and 
several more* The main questionnaire contains over 30 major skip 
patterns (*se table below). The appendix to this Sbctirn 
contains a list cf the major skip patterns anw data on various 
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Filter 


Target 
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sxsxxxxxxxxssx 
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* Q.5 


Q.14 


Work experience 


Q-6-k 


Q. 1 0 
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Q.7-k 


Q. 1 0 
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Q-8-k 


Q.10 
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Q. 13 
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* Q.14 


Q.15 
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* Q.15 


Q.20 


Por ^fleeoTidarv school *f nj* 
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during last 6 vear s 


Q.17-j 


Q.20 


Second T^ostsee . school 


Q*18-j 


Q.20 


Thl r d ^nfltflpo Q^linnl 
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* Q.23 


Q.27 


School costs 
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Q.24 


Q.25 
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Q.25 


Q.26 


Other source 3 of financial aid 
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Q.26 


Q.27 


Student loan rpnAvmpnt 


Q.30-f 


Q.35 


Second living oar'fcner 


Q.31-f 


Q.35 


Third living partner 


* Q.36 


Q.87 


Number of marria^en 

11 WWt W w 1 W 1 *** w 1 A W W 


Q.46 


Q.C, 


Divorced/ separated x rom first snouse 


Q.50 


Q.82 


Legally divorced from first spouse 


Q.62 


Q.64 


Nature of alimony agreement 


Q.65 


Q.81 


Children with first spoure 


Q.6u 


Q.71 


Nature of custody agreement 


Q.71 


Q.78 


Nature of child support agreement 


Q.82 


Q.87 


Marital partner if ever married 


Q.84 


Q.86 


" Spouse* s" 1ob 


* Q.87 


Q. 106 


Expected number of children 


Q.89 


Q.106 


Parenting history 


Q.98-a 


Q.99 


Care of pre-school children 


Q.99-a 


Q.102 


Care of school age children 


H Q.112 


Q.117 


Respondents ' parents 


Q-i23 


Q.125 


Teaching as a career 


Q. 125-b 


Q.1I6 


Attitudes on teaching shortages 



Items marked with an asterisk (») were asked of all 506 NLS-72 
field test respondents. 
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Ca average, the percentage of various errors is comparable to 
the levels found in the HSAB data. (Since the NLS-72 field test 
sample is significantly larger than the HSAB field test sample, 
the actual number of oases committing errors seems larger.) As 
noted below, tuc patterns of error axe also similar to those 
described above. For certain patterns, the apparent amount of error 
is larger than for the xISAB field test. Ve believe that this is a 
function of the confounding influences of the routing patterns 
with the exceptionally long fol3ov;-up recall period for the NLS-72 
sample (six years) compared to the HSftB sample (two years). 
Although revisions and clarifications to routing instructions 
should help to reduce inconsistencies, the much greater recall 
period for NLS-72 sample members is likely to produce a generally 
higher proportion of missing responses, some of which will create 
inconsistencies I^tween filter and dependent questions. 

Empl ,ment Histories (NLS-72) 

Ninety-six percent; of the NLS-72 field test sample reported 
having at least one part-time or full-time job between October 
1979 and February 1985- Only 1 percent of the respondents failed 
to answer this important filter question. An additional 1 pei .ent 
of the respondents failed to report the type of job th jy hrid in 
Q.7 (the follow-up items on the first job held during the period). 
Only 2 percent of the respondents failed to repor I the months they 
had held their first job in response to Q.7-e. Approximately 
8 percent of the sample failed to report their current (or ending) 
salary for their first job in response to Q.7-h. As we suggested 
in the prior section on the HS&B questionnaire, missing data on 
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this item are more likely to result from recall problems or hidden 
refusals than from lack of clarity in the question format or 
wording. 

Fifty-eight percent of the sample reported holding a second 
job. The proportions missing responses in the parts of Q.8 were 
even smaller than foe the questions on the previous job (if 
calculated as a percent giving inconsistent responses of the whole 
sample). The same results obtained for the 31 percent who 
reported three jobs and the 18 percent who reported holding four 
jobs during the period fro* 1979 to 1965. Levels of response 
inconsistency we* e generally around 2 to 3 percent of the field 
test spjcple. 

The series on employer -provided training (Q.11 through Q.12; 
exhibited the same problems as described for the HS&B 
questionnaire. I; substantial numbers, respondents failed to mark 
the types of training benefits or programs they had experienced, 
even though they did provide the numbers of hours and th*; numbers 
of weeks that the benefit* were received. Ve recommend 
reformatting this item along the lines suggested 'for HSftB above. 

Periods of Unemployment (NLS-72) 

Respondents who reported having held no jobs during the 
period in question were instructed to skip to Q.14- on periods of 
unemployment. A total of 26 <ndlviduals (5 percent of the samp 
failed to correctly follow the skip instruction at Q.5 and omitted 
their response to Q.H. Of the«e, 7 individuals responded to the 
follow-up item the specific months unemployed, ye believe that 
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in such instances an affirmative response to the filter item Q.H 
may be imputed. 

Postsecondary Education (NLS-72) 

In response to filter item Q.15, 193 NLS-72 respondent* 
reported attending at least one postsecondary school since 
October 1979. (Only 66 respondents attended more than one school 
and only 12 attended more than two schools.) About 1 percent of 
the cases failed to report the name of the first school attended; 
about 2 percent failed to mark the calendar in Q.17-b to indicate 
the months attended. Response inconsistency on the schooling 
items hovered at about 1 to 2 percent across all dependent items. 

Respondents were asked a second question on the periods of 
postsecondary school attendance since 1979 in Q.23. In general, 
responses to Q.15 were consistent vith those of Q.23. Abouu 5 
percent gave inconsistent responses (about half gave a positive 
response to 0.5 aiid omitted Q.23; about half indicated a period of 
attendance in Q.23 after having reported no school attendance a*>- 
Q.5). Given the low levels of inconsistency, we do not recommend 
revisions to either item. 

After indicating at least one period of attendance in Q.23, 
about 35 sample members (7 percent of the whole sample, 18 percent 
of those reporting attendance) did not report total costs of 
attending their first school. We believe that the *roblem here is 
merely one of recall of cost data over a six-year period rather 
than a problem of questionnaire design. A similar conclusion 
seems reasonable concerning the 7 percent of cases who failed to 
respond to Q.27, concerning the years the respondents' parents 
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claimed them as exemptions on Federal tax returns. Ve believe 
that this response error results from the difficulty of recalling 
the events in question, not from ineffective routing instructions. 

Family Formation (NLS-72) 

Although most of the items in this section are supported by 
the NIH supplement to the NLS-72 Fifth Follow-up, we believe them 
to be of sufficient interest to include a brief overview of 
response consistency in this report. 

The series of items on family formation behaviors begins with 
Q.30. This item asks for the date when the respondent began 
living with their firut partner in a marriage or marriage-like 
relationship. Because th* wording and format of tuis item failed 
to provide a means for respondents to indicate that they never had 
lived in such a relationship, it is imposaible to determine the 
precise number of cases who should skip over this series of items 
on living arrangements. Field test editors kept track of marginal 
notes written by respoi J ents to the effect that the items were 
"not applicable," or that the respondent had "no such 
relationship." Thirty respondents added such handwritten notes. 
An additional 44 respondents omitted all the items in this series. 
Although these appear as missing responses in the field test data 
files, it remains possible that they would have chosen a negative 
response to a filter item (preceding Q.30-a) on whether they had 
ever lived in such a relationship. Ve propose to include such a 
filter item in the revised questionnaire for the full-scale study 
in order to properly classify all i-espondents and route them to 
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appropriate follow-up questions. In this case, respondents who 
answer the new filter item negatively will b-5 routed to Q,87, 

About 436 cases (86 percent of the sample) indicated 
participation in at least one marriage or marriage-like 
relationship. Eighty-seven respondents (17 percent) reported two 
or more such relationships f and 15 respondents (3 percent) 
reported three or more. 

Fifty cases failed to respond to the item on whether the 
first relationship ended (Q,30-f), However, a small number of 
these cases provided data o\\ the dates the relationship ended, and 
the type of separation (item Q,30-g). Ve believe that missing 
data to Q.30-f could be reduced by reformatting to allow the 
response choices for * ? Ycs" and "No" to crtand out more clearly, 

Q,36 begins a series of items for those respondents who have 
been legally married at least once. Four hundred and seven 
(80 percent) of the sample members indicated at least one 
marriage. Only about 16 respondents fail:, to respond to the 
items on the religious preference, ethnic background and prior 
marriages of their first spouse, Ve believe that missing data on 
these items results from either problems in recalling the relevant 
information, or of refusal to answer, and not from question 
wording or format. 

About 5 percent of the sample failed to respond to Q,45 on 
perceived happiness at the end of each year of marriage. The 
level of missing data is too small to indicate a major difficulty 
with the instructions. In the absence of a more persuasive 
explanation for the missing responses, we will attempt to format 
tiiis item to increase the clarity of the directions. 
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Family Disruption 

Q.4-6 asks respondents to indicate whether their first 
marriage is still intact or has ended in separation or divorce* 
One hundred eleven respondents reported a disruption, virtually 
identical to the number who reported disruption of the.'r first 
marriage-like living arrangement in Q.30-f . There is almost no 
missing data in the items from Q.4-7 through Q.4-9. In item Q.50, 
respondents are asked to confirm that they are legally divorced, 
and new routing instructions are issued. Although 100 respondents 
had indicated that they were legally divorced in Q.4-6, only 85 
explicitly did so again In Q.50. Another 6 cases failed to 
respond to the check box in Q.50, but answered the dependent items 
Q.51 through Q.88, allowing an imputation of their divorced 
status. Nine respondents failed to check the box in Q.50 and 
skipped the subsequent items. Although the reason is not fully 
clear, these few respondents may have skipped the box in Q.50 in 
order to justify taking a shorter route through the remainder of 
the questionnaire. 

There is virtually no missing data for most of the items 
between Q.50 and Q.81 f covering divorce and child support issues. 
One exception is Q.71 concerning the child support agreement. In 
Q.65, 4-7 divorced respondents imported having had children with 
+ heir first spouse. All of these described a custody agreement in 
Q.66 (including the "no agreement" response). Only 21 of these 
respondents indicated in Q.71 the type of child support aggreement 
reached during the divorce, and only 37 reported the amount of the 
payment or its regularity (Q.72). 
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Current marital status is ascertained by Q.82. for those who 
had experienced some form of marital disruption. It is also used 
to direct respondents through a series of items on the employment 
of their current spouse. However, as this section is currently 
arranged, the information on current spouse's job is not collected 
for respondents who are still married to their first spouse. This 
omission results from a mistake in the skip instructions in Q.46, 
which directs respondents to Q.87, but should direct them to Q.83. 
Because of this error, 355 respondents were inadvertently routed 
around 0.8 83 through Q.86. 

Parenting (NLS-72) 

In response to Q.89, 232 respondents indicated having one or 
more children. Some 66 respondents failed to Till in a numeric 
response, leaving the response l,oxes blank. These respondents 
uniformly omitted all remaining items on children, strongly 
suggesting that the blank response implies no children. For the 
full-scale survey, we propose that a limited number of fail edit 
call-backs be made to respondents who leave blank the item on 
number of children. If the results indicate that nearly all (at 
least 85 percent) of the omissions mean that the respondent has no 
children, we will recommend abandoning further retrieval calls and 
instructing coders to enter zeroes to replace blanks. 

The amount of missing data in the series of follow-up items on 
children (Q.s 91-97) is very small. Only 1 percent of field test 
respondents reported having more than 4- children, suggesting that 
Q.s 95 through 97 might be dropped from the questionnaire. 
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About 5 percent of the sample failed to answer Q.99-a on 
the presence school-age children. Although the number is not 
excessive, we believe that it can be reduced further by 
reformatting this item (and the prior it en. on preschool-age 
children) along the lines desc. id for the questions on employer- 
provided training. That is, we believe that the filter item 
determining the presence of pres.chool- or school-age children 
should be made more prominent in the item format. 

Only 4- respondents who reported having children failed to 
answer the item (Q.104-) about educational aspirations for those 
children. In addition, respondents with two or more children did 
not appear to have much difficulty giving a single response to 
this item, which asks about "all or most of your children." Only 
7 caser ga^e multiple answers to 3.1 04-. 

Respondents who indicated at Q.87 that they expected no 
children, and those who reported at Q.89 that they had no children 
were instructed to skip to Q.106 on the composition of their 
families when they were seniors in high school. Eighteen 
respondents (4- percent) failed to answer Q.106. H< ever, 11 of 
these respondents had also failed to answer Q.89 as well. 
Furthermore, there appears to be a minimum of 11 cases who failed 
to answer all .terns from Q.106 to the end of the questionnaire. 
These inconsistencies do not result from problems with question 
wording or format, but simply from nonresponse to the latter 
sections of a comparatively long questionnaire. The fact thit 
only 3 to 4- percent of respondents failed to answer these items 
testifies to the overall soundness of the questionnaire as a aelf- 
administered survey form. 
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Attitudes towards Teaching as a Profession 

In the final section of the NLS-72 field test questionnaire, 
respondents were asked a series of items on their interest in 
teaching as a career, and the issues influencing their views of 
the teaching profession. The first filter item in this series 
asks whether respondents ever considered teaching as a career. 
Twelve respondents omitted this filter item. A total of 194 
respondents (about equal to the number selected from the NLS-72 
sample for this field test as actual or potential teachers) gave a 
positive response and were directed to Q.123-a. The 49 
respondents who reported that they were currently working as 
teachers were directed to answer Q.123-a, and then to proceed to 
Q.123 ~ skipping over Q.123-b and Q.124, items intended for non- 
teachers on?.y. Altogether, 29 oi' the respondents who said that 
they had considered teaching as a career failed to answer Q.123-a 
on the importance of various factors to their considerations. 
Nearly all (24) of the inconsistent respondents ind* ated at Q.123 
that they were currently working as teachers. Thus, half of the 
current teachers apparently did not detect that the routing 
direction included two instructions: "ANSWER A AND SKIP TO Q.125." 
This problem ^an be alleviated by elaborating the skip instruction 
more fully us '.ng such wording as "FIRST ANSWER 0.123-A BELOW, THEN 
SKIP TO Q.125 ON PAGE _." As an added precaution, an extra line 
might be insrrted below imt item of Q.123--a that reads: "IF YOU 
ARE CURRENTLY WORKING AS A TEACHER, SKIP TO Q.125 ON PAGE _. » 

Items comprising Q.123~b were omitted by between 33 and 45 
respondents. Nearly all of those who failed to answer had 
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indicated at Q.123 that they had never considered teaching as a 
profession. As above, we believe that clarifying the routing 
instruction might reduce substantially the number of missing 
responses. The instruction for this response option will be 
altered to read: "SKIP TO Q.123-B ON PAGE Furthermore, we 

believe- at confusion can be reduced if each of the groups of the 
items in the Teaching Section of the questionnaire is given a 
separate item number. (For example, Q.123-a would become Q.124; 
Q.123-b would become Q.125, and so on.) We note, however, that 
there are 18 to 20 respondents who omit all items after Q.123 in 
the questionnaire. It thus remains unclear whether the missing 
data is closed by unclear routing instructions or respondent 
fatigue. Nevertheless, we pr^poee to make the changes described. 

In response tc Q.124-, 136 respondents indicated that they 
would consider going into elementary or secondary school teaching 
"if the conditions were right." Twenty-one cases omitted this 
filter item. In the follow-up series, there was virtually no 
missing data for the 136 potential teachers. About one third of 
those who omitted the filter (Q.124) gave responses to the 
follow-up items. 

No response inconsistencies were observed between Q.l25-a and 
Q.125-b. However, 34- respondents vho said they felt that teacher 
quality was a problem in elementary and secondary schools at 
Q.125-a failed to respond to the set of follow-up items under 
Q.125-C This problea clearly stems from the potentially 
inconsistent instructions some respondents receive who answer 
"Yes" to Q.125-a, but who answer "No" or "Don't Know" to Q.125-b. 



9 

ERIC 



22 

79 



The latter group are instructed to "SKIP TO Q. 126, 11 contradicting 
the prior instruction to "ANSWER B AND C" associates with their 
"Yes" answer at Q.125-a. To eliminate this confusion, NORC 
recommends removing the individual routing instructions from the 
response options to Q.s 125-a and 1 25-b and replacing them with 
an instruction box that reads: IF YOU ANSWERED "YES" TO EITHER 
Q. 125-A OR Q.125-B, GO TO Q.125-C ON THE NEXT PAGE. OTHERWISE, 
SKIP TO Q.126 ON PAGE (In the final version, the item numbers 

will be changed to simpler numbers, without the sub-part 
designations. ) 

Teaching Supplement 

There are relatively few routing patterns employed in the 
Teaching Supplement, and those that are used are comparatively 
minor in scope. These will be discussed in a separate section 
devoted to a general analysis of the Teaching Supplement 
Questionnaire. 
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NOTES- Nl is the total number of respondents who should have answered 

each filter item. N2 is the number whose response to the filter 
item makes them eligible to answer the dependent items associated 
with the specified filter Question. 

"Omitted Filter* indicates the number of cases coded as 
•illegitimate skips" on the filter item. If these respondents 
answer the dependent questions in specific ways, it it: sometimes 
poacible to impute what their responses to the filter item would 
ha 4 -j been. 

The column labelled -"Depndt M contains the question number of 
asample of the dependent items associated with the specified 
filter question. 

Two types of errors are counted in the n^<t two co 1 '»nz-. errors 
cf or.is.sion ("Omissn") occur wh*n a .respondent should answer the 
'-impendent item but does not; error* of commission < "Commissn" > 
occur when the respondent provides an answer to a depcnJcndent 
question aft~r having been instructed to skip over it. 
The next column < "Impute Filter") counts the number of cases 
whose response to t.i* depf-nde^t items would allow imputation or 
missing data for the associat J filter. 

Ths next twr columns contain the question number for thL "Targot" 
item for eacn filter, and the number of cases who fail to answer 
the designated target item. 
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4. EVENT HISTORY ITEM FORMATTING (CALENDARS) 

The field test provided an opportunity to test a new format 
for questionnaire items about periods of employment, unemployment, 
and postsecondary school attendance. Respondents were asked to 
circle codes printed in a calendar-style lascut that corresponded 
to months and years during which they may have been employed, 
unemployed, and/or enrolled in school. In past surveys, 
respondents were usually asked only for the beginning and ending 
dates for t>:ose events. The calendar format was developed to aid 
respondent recall of time periods, to better capture interrupted 
spells of schooling and work, and to simplify the method of 
reporting this information. 

Recall and reporting burden were of particular concern for 
the NLS-72 respondents because they were asked about employment, 
educational experiences, and family formation and dissolution for 
a six-year period. During previous follow-ups, NLS-72 respondents 
reported periods of employment and school attendance by writing in 
the beginning and ending dates of each episode. Entering dates 
for the number of employment and educational situations called for 
in the field test questionnaire would have been a considerably 
larger task than it had been during prior follow-ups, however, 
because of the longer period of time covered. 

Previous HSftB questionnaires had included some items with 
formats that were similar to the calendar layout. Answer grids 
ha*' been used because questionnaires were optically scanned during 
the data preparation process. Respondent performance on the 
gridded items had indicated that tne format did not pose any 
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significant difficulty, and therefore calendar format items vere 
not expected to create any special problems. 

The following evaluation of calendar items is based on 
frequency distributions of field test data and on a qualitative 
evaluation by an NORC coding staff who edited tteld test 
questionnaires prior to data entry. Frequency distributions 
indicated the overall level of item nonresponse for these new 
format items, and the qualitative assessment provided a sense of 
the typeu of difficulties respondents had in using the calendar 
items. Coders were asked to note th* ability of respondents to 
follww instructions for completing the calendars and patterns of 
inconsistency between information given in calendars and data 
supplied in other items- Coders were able to note problems 
obvioua only by visual inspection, such as whether circles were 
drawn clear ly, and also to note comments that respondents had 
written in questionnaires. 

Table 4-1 provides a summary of item nonresponse for the 
calendar items- The overall level of item nonresponse, which does 
not exceed 4-0 percent, is within an acceptable range so that 
statistical analysis vould not be tampered. The table also shows 
nonresponse rates that were calculated for just those respondents 
who were eligible to complete the calendar terns based ; answers 
to filter questions. This nonresponse rate is auch greater. The 
rate indicates that nonresponse tends to increase with the nun »r 
of jobs and schools that respondents report. For example, the 
proportion of HS&B respondents who were eligible to complete the 
fourth job calendar but failed to do so (14.0 percent) was over 
three times the proportion of eligible respondents who did not 
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Table 4-1 

Respondent Performance on Calendar Format Items 







Item 






Number of 


nonresponse 






eligible 


rates for 


Overall item 




respordents 


eligible 


nonresponse 




not responding 


respondents* 


rates** 



HS*B job calendars 
1st job 
2nd job 
3rd job 
4th job 

HSftB school attendance calendars 
1st school 
2nd school 

HSAB unemployment calendar 



12 
7 

1 1 
7 



NLS-72 job calendars 

1st job 8 

2nd job 12 

3rd job 15 

4th job 12 

NLS-72 unemployment calendar 18 

NLS-72 school attendance calendars 

1 st school 1 1 

2nd school 7 

3rd school 7 



3.8* 
3.6< 
10.7* 
14.0* 



3.9* 
18.5* 



3.5* 



1.6* 
4.0* 
8.5* 
10.5* 

11.0* 



5.6* 
9.5* 
35.0* 



,6* 
,1* 
3* 
1* 



2.1* 
3.0* 

1.2* 



1.6* 
2.4* 
2.9* 
2.4* 

3.6* 



2.2* 
1.4* 
1 .8* 



♦Percent based on respondents eligible to answer on the basis of 
responses to filter questions. 
**HS4B rates based on 334; NLS-72 rates based on 506. 



complete the first job calendar (3.8 percent). A comparison of 
NLS-72 nonresponse for the same items indicates a similar increase 
in nonresponse rates from the first job calendar (1.6 percent) to 
the fourth (10.5 percent). A fourth job would have been held more 
recently than the first, and presumably, the months during which 
it was held would be at least as easily remembered. The pattern 
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of nonresponse suggests that survey members may become less 
diligent and thorough in completing items toward the end of long 
or complex series* 

The coder evaluation of questionnaire items with calendar 
formats tended to support the idea of respondents' decreasing 
diligence. Coders reported that circling individual months seerced 
to be a tedious task that some respondents attempted to shorten in 
a variety of vays. For instance, they circled first and last 
months, blocks of months, names of months and yea s, or added the 
notation "all" indicating all months. Some respondents also 
commented in questionnaires that circling months was tiresome. 

A possible remedy for high nonresponse rates due to question 
format is an alternative format. The format that was used during 
previous follow-ups for event history information is one that 
requires respondents to write in beginning and ending dates.. It 
is possible, however, that such a format (which does not provide 
the visual cues offered by the calendar layout) would result in an 
even higher nonresponse rate— particularly for the NLS-72 
participants who have to give information about a six-year period 
of time. Therefore, using an alternative format that might be 
more familiar to respondents does not offer a risk-free solution 
to the "\mresponse problem. 

The nonresponse rates for the calendar format items do not 
appear to have serious implications for the costs of collecting 
data during the full-scale survey. Questions about periods of 
employment , unemployment , and school attendance are expected to be 
critical items as they have been during previous follow-ups. If 
respondents do not complete calendar items or fail to do so 
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correctly, they will be contacted to supply or clarify 
information- It should be noted that although thare were high 
nonresponse rates for some of the calendar items, the rates were 
attributable to relatively small numbers of respondents- as the 
table indicates, for example, there were only 10 HSJfcB respondents 
responsible for the highest nonresponse rate on any of the HSJLB 
calendars (i.e., 18-5 percent for the second school calendar). 
There were only 7 NLS-72 respondents responsible for the 35-0 
percent nonrisponse rate for the third school calendar in the 
NLS-72 questionnaire. 

The impact of nonresponse on data retrieval costs during the 
main survey for these items should not be great since the number 
of survey members involved will be relatively suall. There are n< 
obviously preferable alternatives tc ae calendar format, and it 
does seem to be workable for most respondents. Based on these 
considerations, it is recommended that the calendar format be 
retained. In the final preparation of the questionnaire, 
attention will be given to instructions or aids that will 
encourage respondents to be thorough in completing calendar items 

Although the calendar format is not expected to seriously 
affect data collection costs, it will have a very significant 
effect on data storage costs and on the costs and ease with which 
data can be manipulated. The calendar format requires a separate 
variable for each month of each year. The seven HS&B calendars, 
each covering twenty-four months, require 168 variables. There 
are 669 variables needed for the nine NLS-72 calendars (eight of 
which cover 65 months each, while the ninth covers H9 months). 
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To record data for meat individual respondents, only a portion of 
the total number of variables are need. For the majority of 
respondents, most of the data points that correspond to the 
calendar months are empty. The total number of variables for the 
NLS-72 questionnaire, which is greatly increased by the calendar 
items, exceeds computer software package limitations for one file. 
The two files that are consequently needed for questionnaire data 
are less convenient, less efficient and, of course, more expensive 
to use and maintain. 

The calendar format was intended to offer an effective way of 
capturing periods of interrupted schooling and employment. 
Further qualitative assessment of field test questionnaires will 
be made to determine if the format was successful in -doing so. 
If additional evaluation indicates that objectives were met, that 
is, that the format identifies interruptions, the analytical needs 
of those who will eventually use the survey data may best be 
served by maintaining separate variables for each month. If it is 
found that calendars did not fulfill their purpose, 
recommendations will be made for data preparation procedures th,-vr. 
will simplify data entry and reduce data file size. For 
instance, coding specifications can be written so that only 
beginning and ending months will be data sntered rather than 
values (or missing data codes) for all months covered by each 
calendar, or, if respondents indicate multiple, interrupted spells 
of schooling or work in the calendars, two to four sets of 
beginning and ending dates could be coded for each event. 

On the whole, relatively few interrupted calendars were 
found. la more than half of a sample of 100 field test 
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questionnaires that were reviewed, there were none at all. Only 
one interrupted calendar was found in 30 percent of this sample of 
questionnaires, and the renaming 20 percent tended to contain 
only two or three interrupted calendars. Interruptions were 
reported most often for first job, first school attended, and 
living partner arrangements, m& maintaining separate month 
variables for these calendar items might be desirable. 

Although a more thorough analysis of these items will be 
undertaken, the evidence appears to support a decision to code the 
beginning and ending dates of spells reported on calendars, or at 
least those calendars which tend to be uninterrupted." This 
would serve to reduce the size of the main survey data file 
without compromising data quality. 
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5. NEW QUESTIONNAIRE ITEMS — HS4B CORE QUESTIONNAIRE 

One of the principal values of a longitudinal research design 
is the ability to compare responses to common survey measures of 
data from wave to wave. This requires using identical or nearly 
identical questionnaire items for each survey. There have been 
few changes in the HS&B instrument since the last follcv-up. The 
scope axsd structure of the questionnaire have remained largely 
unchanged. Several questions, such as those about military 
experience, have been removed- With the exception of the four 
questions that were added and are reviewed in this section, and 
several for which forma%s were modified (discussed in "Event 
History Item Formatting" ) , changes have been limited to minor 
rewording or revisions of dates. 

The following evaluation of new HS&B questions was based on 
analyses of frequency distributions and on the observations made 
by the NORC coding staff who edited all questionnaires prior to 
data entry. 

Question 15 (New) 

Respondent* were asked to think about their present or most 
recent job and to indicate whether a list of 3t\*ements describing 
job characteristic were true or false in terms of that job. 
Statements for this question were taken from the Monitoring the 
Future survey base year and follow-up questionnaii es (Form 4, 
Part A), cor-* Dted by the Institute for Social Research at the 
University of Michigan. 

The nonresponse rates for these items were lov, and 
participants seemed to have had no difficulty responding to any 
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statements. Frequency distributions (shown below) reflect a 
satisfactory amount of variability in respondents' answers. No 
changes are recommended. 

Question 15. Thinking about your present or most recent job, 
indicate whether the following statements are true or false. 
(CIRCLE ONE FOR EACH LINE) 

True False 

a. It is (was) an interesting job to do 87* 13* 

b. It is (was) a job I could be happy doing 

for most of my life 40 60 

c. It is (was) the type of work I expect to 

do for most of my life .....32 53 

d. It is (was) a good stepping stone toward the 

kina of work I want in the long run 59 41 

e. It is (was) the kind of work people do just 

for the money 54 65 

f. It causes (caused) me stress and tension 4.5 55 

g. It interferes (interfered) with my education 19 81 

h. It interferes (interfered) with my social life 27 73 

i- It interferes (interfered) with my family life 24 76 

j. It lets (let) me get to know people with social 

backgrounds very different from mine 73 27 

Question 17, 18, and 19 (New) 

These questions ask about how prepared respondents were to 
handle the daily work activities, and about employment absenteeism 
and tardiness. The frequency distributions for these items are 
shown below. 

Responses to these items seem t( e in keeping with the 
general level of satisfaction reflected by other questions 
about current (or most recent) employment situations. For 
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instance, most respondents (82 percent) indicated they were either 
satisfied or very satisfied with their jobs as a whole (Question 
Hj). Responses to Question 17 indicate that only 7 percent of 
the HSftB respondents were usually or fairly often unprepared to 
handle activities at vork, wuestion 18 responses suggest that 
absenteeism was low (90 percent were absent four days or less 
during the last six months). Based on the answers to Question 19, 
most were prompt (85 percent were late four days or less). 

The nonresponse for these items was low (less than 
1 percent); and based on the coders' review of questionnaires, 
respondents did not seem to have any difficulty answering the 
questions. No changes are necessary. However, given the very low 
incidence of extreme absenteeism and tardiness , we ould recommend 
reducing the number of categories in Questions 18 and 19 by 
combining the last three into a Tingle optic* that reads, "11 or 
more days" or "more than 10 days." 

Question 17. Thinking about your present or most 
recent job, how often do you find yourself coming tc 
work unprepared to handle the day's activities? 



Usually 1* 

Fairly often 6 

Seldom 60 

Never 34 



Question 18. During the LAST 6 MONTHS (or, if not 
currently working the last 6 months that you worked), 
about how many days were you absent from work for any 
reason. Do not include vacations or jury uaty. 



None 35* 

I or 2 days 39 

3 or 4 days 16 

5 to 1 0 days 5 

II to 15 days 2 

16 to 20 days 1 

21 or more 2 
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Question 19. During the LAST 6 MONTHS (or, if not 
currently working, the last 6 months that you worked), 
about how many days were you late to work? 



None 41* 

I or 2 days 29 

3 or 4 day 8 15 

5 to 10 days 8 

I I to 1 5 days 2 

1 6 to 20 days 1 

21 or more 3 
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6. NEW QUESTIONNAIRE ITEMS — NLS-72 CORE QUESTIONNAIRE 

Many of the NLS-72 questions in the field test instrument 
were repeated from previous questionnaires. Changes that were 
made in the questionnaire for this follow-up were primarily 
additions of new questions and minor modifications of previoucly 
used items. New questions were added so that NLS-72 data could be 
used to address additional education policy issues and social 
research interests. Most modifications were made so that 
questionnaire items (including wording, instructions , format, and 
answer categories) would correspond exactly to items in other 
survey instruments, primarily previous NLS-72 survey forms and the 
HS4B questionnaire, thus enhancing the comparability of similar 
items from separate studies. 

The evaluation of new and revised items is the focus of this 
section. These items have been examined in several ways. 
Frequency distributions were used to evaluate their effectiveness. 
Distributions of participants 1 responses helped determine if 
instructions were clear, if the intent of questions was met, and 
if response categories were comprehensive. A qualitative 
assessment of new and revised items was also made. NO^C coders 
reviewed all questionnaires and noted patterns of errors mr ie by 
respondents in filling out questionnaires. The comments that 
respondents had written in questionnaires were also noted. 
Participants had been encouraged to identify questionnaire items 
they found problematic and to explain why these were difficult to 
answer (e.g., instructions were not clear, questions were not 
applicable because of respondents* circumstances) . 
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The questionnaire items that are reviewed in this section are 
in numeric order; numbers correspond to the field test 
questionnaire. Each item has been identified as new or modified. 

Question 12 (New) 

Respondents were asked if training they had received as part 
of their most recent full-time job had been specifically aimed at 
retraining thsm because their current job was phased out or 
discontinued. 

There was a high nonresponse rate for this item 
(13.8 percent) which is likely to be attributable to its position 
on the questionnaire page and to its placement in relation to a 
series of questions asking for detailed information. This item 
appeared at the bottom of a page and after a series of questions 
asking for estimates of time (in weeks and hours per week) spent 
in various job training situations. It is likely that the 
comparative demand of the preceding series of questions eclipsed 
this item, and that response would be improved by more prominent 
placement. An alternative would be to incorporate it as part of 
preceeding series. Refer to sections on "Nonresponse and 
Respondent Burden" and "Logical Consistency between Filter and 
Dependent Items" for additional discussion of this item. 

Question 1 3A-B (New) 

Part A of Question 13 asked if respondents had received 
training or education to improve skills c. to advance within the 
job they had held. Part 3 asked if they had eceived training to 
prepare for a new job or career. 
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The nonresponse for both parts of this question was 
comparatively high (7 percent and 9 percent) though not as high as 
for Question 12. It is likely that placement of these questions 
(they appear at the top of a page rather than at the bottom as 
Question 12 did) accounts for the somewhat better performance. 
Response capture might be further improved if these items were 
moved to precede the detailed job training series that they 
followed in the field test questionnaire. Because the nature of 
inquiry is broader than the job training series, document flow 
might also benefit from placing the more general questions before 
those that require specific, quantitative information. This item 
is also discussed in the "Nonresponse and Respondent Burden" 
section. 

Question UB (Modified) 

The format of this question, asking about periods of 
unemployment, haa been modified. In previous questionnaires 
respondents were asked to give beginning and ending dates to 
report when they had been unemployed. The revised format was a 
calendar-style layout that enabled respondents to circle codes 
that corresponded to months and years rather than write in 
information. The nonresponse rate for this item was comparatively 
high: 11 percent of respondents who had experienced unemployment 
failed to complete the calendar. It is likely that nonresponse is 
attributable to recall difficulty rather than to the new format. 
NLS-72 survey members are asked to give unemployment information 
for a six-year period of time. It is not expected that the rate 
of nonresponse for this item will have a significantly negative 
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effect on data retrieval costs since the number of survey members 
who contribute to the rate is small. No changes are recommended. 
See the "Event Histc-y Item Formatting 

(Calendars)" section for a full discussion of the calendar format 
items. 

Question 23, 23A, 23B; Question 24, 24A, 24B; and Question 25, 25A 
(New) 

This series of questions gathers data about postsecondary 
school finances. Although NLS-72 respondents had been asked about 
their school finances during previous follow-ups, these specific 
questions had not been used before. Survey members were asked to 
supply information about the two most recent time periods between 
October 1979 and February 1985 during which they attended school. 

There was a high level of nonresponse for these questions 
which suggested that respondents found it difficult to keep time 
periods and corresponding financial information clearly in mind. 
Nonresponse rates indicated that identifying specific types of 
financial assistance was also difficult. 

Please refer to "Nonresponse and Re3pondent Burden" for a 
full discussion of these questions and recommended modifications. 

Question 98, 98A, and 98B (New) 

Respondents were asked to select a response category that 
described the child care arrangements they made for their pre- 
school childien. Parts A and B of this question ask for estimates 
of days per week and average number of hours per week during which 
their children were cared for in the manner they specified. 
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The nonresponse rates for these items ranged from about 
20 percent of respondents who were eligible to answer to over 
28 percent. For the most part, the high nonresponse rate results 
from an imprecise filtering of respondents into the series of 
questions about child care. The general instructions at the 
beginning of the section direct all parents to answer all 
questions regardless of applicability to their particular 
circumstances. 

Answering tne&e questions was clearly problematic for "full- 
time" parents, that is, parents not employed outside the household 
who them3elves provided all care for their children. Several 
respondents commented in marginal notes that they were not certain 
how to differentiate between "child care" (a term that connotes 
care under certain circumstances, usually the absence of a parent) 
and the general, round-the-clock care that they provided for their 
children. Although one of the response categories was 
satisfactory in describing the care their children received ( M In 
child's home by relative, e.g.* mother, father, grandparent, aunt, 
sibling"), they were not able to give a meaningful estimate of 
child care time per week. Some who attempted to estimate entered 
the maximum number of hours permitted by the format of the 
question (i.e., 99 hours) and added the notation indicating 
that care exceeded 99 hours. The questions seemed to be 
particularly awkward if children were infants. 

An explicit response category should be added to this item to 
allow parents to indicate that they provide all care for their 
child(ren). The option should carry instructions to skip to the 
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next question without entering any further estimates of days and 
hours per week. 

Question 98C (New) 

Respondents were asked about the extent to which they made 
arrangements for child care for their pre-school age children 
(that is, did they make all, most, about half, some or none of the 
arrangements). This item was also awkward for full-time parents, 
particularly those with infants. An answer category such as 
"child care arrangements not necessary" should be added. 

Based on marginal notes in questionnaires, the meaning of the 
term "arrangements" was not clear to some respondents. One 
possibility a respondent mentioned was the initial arrangement 
made for child care services as distinguished from the continuing, 
perhaps daily arrangements that might be necessary (such as 
dropping off and picking up children). One parent noted that 
although his wife had made all of the initial arrangements for 
child care, he was solely responsible for making daily 
arrangements to tranport childrzn to and from the child care 
location. 

The intention of this question is to determine if the 
respondent bears the burden of responsibility for child care 
arrangements and if such responsibility interferes with his or her 
education or employment obligations. The question might be 
reworded in the following way to shift the focus from the kinds of 
arrangements to the locus of responsibility: 
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To what extent do you have responsibility for the 
child care arrangements for your preschool children? 

1. Childcare arrangements not necessary 

2. I make all of the arrangements 

3. I make most of the arrangements (more than half) 
*. I make about half of the arrangements 

5. I make some of the arrangements (less than half) 

6. I personally make no child care arrangements 

Question 99, 99A, 99B, 99C (New) 

This series of questions duplicates Question 98 hut asks 
about child care for school-age children. The questions posed the 
same problem for full-time parents. A similar solution, i.e., 
providing an opportunity for full-time parents to skip, should be 
implemented. Respondents noted that these questions were awkward 
if children were adolescents. (Although members of the NLS-72 
cohort are in their early 30s, some now have partners with pre- 
teen or teenage children from previous relationships.) Adding a 
response category such as "child is able to care for self" or 
"child old enough to care for self" and allowing the respondent to 
skip to the next question would further improve this item- 
Question 102 (New) 

Question 102 asks respondents about the amount of time they 
took off from work when their youngest child was born. 

It is likely that the high nonresponse rate for this item 
(17 percent) is attributable tc parents who were not employed at 
the time the child was born. This might often be the case for 
mothers who stopped working several weeks or months before the 
expected date of birth. An answer category such as "Was not 
working" (placed at the top of the list) should be added for this 
question. 
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Interestingly, aiore respondents (4-2 percent) selected the "No 
time off" answer category than any other. This distribution of 
responses may be due to male respondents who would be less likely 
to take time off from work when children were born. 

Question 103 (New) 

Respondents were asked how much time they had taken off from 
work or school during the last year to care for children. As for 
the previous question, there was a high nonresponse rate 
(19 percent), and adding an inivial response option such as "Was 
not working or attending school" seems appropriate. The modal 
response category for this item was also "No time off" (4-3 percent 
of those who answered the question), again probably reflecting the 
responses of male parents. 

Question 112 A-D (New) 

Respondents were asked to give their mother's age (or if 
deceased, age at death) and father's age (or age at death if 
deceased). The frequency distributions for Parts A and C of this 
question indicate that respondents were able to supply the ages of 
their parents. Moreover, the relatively minor degree of response 
clusters at such "rounded numbers" as 4-5, 50, 55, and 60 reems to 
indicate that respondents are providing reasonably precise data, 

Nonresponse for Parts B and D (those who indicated a parent 
was deceased) was very low at only 9 to 13 cases, (Only 34- cases 
(7 percent) indicated that their mothers were deceased. Nearly 1 
in 5 cases reported that their father was deceased. These gaps 
may be further reduced by adding a phrase like "If you can't 
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remember exactly, please indicate about how old" and "about which 
year . " 

Questions 117 and 118 (Modified) 

The purpose of these two related items is to collect more 
detailed and accurate data on the race and ethnicity of NLS-72 
respondents. In earlier rounds of NLS-72, respwxxdents were asked 
to describe themselves in terms of general racial categories which 
resultei in an undercount of Hispanics. Question 117, asking 
aoout ethnicity, and Question 118, asking about race, were 
developed for and are used in the HSAB questionnaire. 

The rate of nonresponse for this Question 117 (3.4- perceut) 
indicates that most respondents had no difficulty in describing 
themselves in terms of the ethnic categories listed. 
Approximately one-third of those who contributed to the rate of 
missing data for this question (by using response categories 
incorrectly) jere respondents of non-Hispanic origins who 
attempted to use the last response category "Other Latin American, 
Latino, Hispanic or Spanish descent (WRITE IN)" to indicate ethnic 
backgrounds such as Irish, German, or Italian. A minor change in 
the layout of response categories is recommended. A more obvious 
visual separation of the two main ethnic categories (as shown 
below) should make it clearer that the "Other ... (WRITE IN)" line 
is to be used only by Hispanics. 
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117. What is your origin or descent? (If more than one, 
please mark the one you consider the most important 
part of your background.) (CIRCLE ONE) 



NON- HISPANIC 01 

HISPANIC OR SPANISH: 

Mexican, Mexican-American, Chicano 02 

Cuban, Cubano 03 

Puerto Rican, Puertorr iqueno 04 

Other Latin American, Latino, Hispanic 

or Spanish descent (WRITE IN)__ 05 

Item nonresponse for Question 118, which asks for the 
respondent's race, was only 1 percent. These two questions appear 
to work as intended to better identify Hispanics who belong to all 
racial categories. 
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7. RESPONDENT LOCATING 

The current follow-up surveys of HS4B and NLS-72 are 
conducted primarily by mail. Updating the name and address 
information of participants so survey material will be received 
promptly is an essent<al task in preparing for data collection. 
This preparation, referred to as locating, involves contacting 
participants to verify or correct name and address information. 
Tiie ..ontact also provides an opportunity to renew respondents' 
interest in the study and their willingness to participate. They 
are reminded of the importance of their role in on-going policy 
research and provided with updated information about the project. 
Participants are informed, for instance, about survey findings 
that have become available since they we-e last contactea, uses to 
which data have been put, and plans for future data collection. 
They are also reminded of the nature and objectives of 
longitudinal research and how crucial their participation is to 
the success of the study. 

The level of effort required to contact individual 
respondents during the locating period varies. While name and 
address information for many participants remains the same, 
extensive tracing is required for survey members who move between 
survey waves. The success in eventually contacting respondents 
varies, as does the effort needed to do so. Locating success is 
affected, of course; by mobility behavior of individuals. How 
frequently they move, how far they move, and wh- , kind of 
forwarding information they leave behind will have direct effects 
on the ability to trace them. Locating success is also affected 
by the ability and willingness of participants, family members, 
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friends, ar.d acquaintances to provide information about 
respondents' whereabouts. 

Different locating methods were used for the HSAB an: NLS-72 
cohorts during the field test in order to meet the field test 
objectives for the two components. The field test objectivi* for 
the current HSAB round were essentially those of the lest ro>md— 
the assessment of new and revised questionnaire items. Because 
the primary concern was instrumentation and because the survey 
procedures used previously had been satiE tory, no changes were 
made in locating activities. As during earlier follow-ups. the 
initial prefield contact with the HSAB field test sample was by 
mail. Letters were sent to survey member informing theu of the 
survey schedule and requesting them to mail back updated name and 
address information. Follow-up of those who did not respond and 
of those whose letters wt e returned as undeliverable began about 
two weeks after the initial mailing. In order to compare costs 
and effectiveness of two methods, telephone follow-up was used for 
half of the nonresponders and mail follow-up for the other half. 
The results are described later in this section. 

The field test objeciives for NLS-72 were somewhat broader 
than thotie x'or HSAB and conditioned by the history of survey 
events during the last six years. The evaluation of 
questionnaires was of mad or intersst, but equally important was an 
assessment of the locating effort that would be required during 
the main survey. Unlike HSAB, NLS-72 has not involved data 
collection since 1979. Furthermore, although, address updating by 
mail (similar to the procedure used for HSAB) was undertaken in 
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1981 and a^ain in 1983, there had not been extensive tracing of 
survey members who failed to return updated information. It was 
expected that both locating and persuading NLS-72 survey members 
to participate would be more difficult because of this time lag. 
Although the primary locating method to be used during the main 
survey will be address updating by mail, locating for the ?ield 
test was done by telephone. Telephone tracing, a more expeditious 
locating method than regular postal service, was used because of 
the time limits of the prefield period. Telephone locating also 
offered the advantage of personal contact with respondents after 
an extended period between survey waves and an opportunity to 
gauge informally interest and inclination to participate in the 
next round of survey activities. 

7.1 HS&B Locating Procedures and Results 

Locating letters were sent to HSftB sample members on 
October 19, 1984. Enclosed with the letter was a newsler.ter, an 
address update fo*" and a return envelope. Sample members were 
asked to verify or correct name and address information and return 
the form to NORC. Almost half (48.? percent) of the HSftB survey 
members returned the initial address update form. The follow-up 
of survey members who did not return an update form began in early 
November, about two weeks after locating letters had been mailed. 

As an experiment, two methods of follow-up were used to 
contact those who did not return address update forms. Telephone 
follow-up of approximately half of the nonresponders b*g3* in 
early November. In mid-November, the other nonrespcaaers were 
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sent a second locating letter and requested to return an address 
update form. 

Th4 two methods of follow-up were not equally effective , as 
Table 7-1 indicates. Of the survey members who were sent a second 
locating letter, 44.6 percent sent back address update forms. Of 
the cases for whom follow-up by telephone was attempted, contact 
was made and addresses updated for 81.6 percent. 

Tcble 7-1 

High School and Beyond Third Follow-up Field Test 
Comparison of Locating Follow-up Methods 



Type of Fol low-Up 



Sophomores 

Seniors 

Total 

* of % of 

. follow-up sample 
type (N=408) 



Total 


designated for follow-up: 


91 


90 


181 


(100.0) 


(44 


• 4) 


Total 


designated for mail follow-up 
2nd mailing - no response 
2nd Address Update returned, 
same address 


46 
22 
7 


37 
24 
6 


83 
46 
13 


(100.0) 
(55.4) 
(15.7) 


(20 
(11 
(3 


.3) 

T * 
• / 

2) 




2nd Address Update returned, 
new address 


17 


7 


24 


(28.9) 


(5 


9) 




Total located bj- mail follow-up 


24 


13 


37 


(44.6) 


'9 


1 ) 


Total 


designated for phone follow-up 
Phone follow-up of nonresponders, 
not located 


45 
7 


53 
11 


98 
18 


(100.0) 
(18.4) 


(24 
(4 


0) 
4) 




Phone follow-up of nonresponders, 
same address 


19 


23 


42 


(42.8) 


(10 


3) 




Phone follow-up of nonresponders, 
new address 


19 


19 


38 


(38.8) 


(9. 


3) 




Total located- by phone follow-up 


38 


42 


80 


(81 .6) 


(19. 


6) 



Follow-up efforts resulted in name and address updates for 
close to an additional 30 percent of the HSftB field test sample. 
(This is 30 percent in addition to updates made as the result of 
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the initial locating letter.) Mail follow-up located 9.1 percent 
of the sample, and 19-6 percent were located with telephone 
follow-up 

Overall, approx? uately 82 percent of the HSftB field test 
sample was located during the fall of 1984 (see Table 7-2)- There 
was almost no difference in the percent of each cohort located: 
82.1 percent of the sophomores were located and 81.0 percent cf 
the seniors were found. Differences in the locating outcomes for 
the two cohorts during the current field test were even smaller 
than during the last follow-up in 1982. The rates for sophomores 
and seniors during the second follow-up were 63.7 percent and 69.6 
percent, respectively. The overall percent located during the 
current field test was higher because of the additional follow-up 
activities that were undertaken. 

A comparison was made of the address information 1;hat was 
supplied by those who responded to the initial request for name 
and address information and those who responded as the result of a 
follow-up request. This comparison, summarized in Table 7-3 
suggests a relationship between the likelihood of responding (or 
responding without prompting) and mobility. 

Of those who sent back updated information in response to the 
initial letter, 61 percent lived at the address that was on record 
from the previous field test survey in 1983» and 39 percent liv6d 
at a new address. 

There were 117 survey members who did not respond to the 
first request for name and address information but did provide 
information during the follow-up contact. Of those who responded 
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Table 7-2 

High School and Beyond Third Follow-up Field Tesc 
Locating Outcomes 



Sophomores Seniors Total 
(N=207) (N*201) (N*408) 



Address Verified 


N 


Percent 
of 207 


N 


Percent 
of 201 


N 


• Ci WCIl l 

of 408 


1st Address Update returned, 
same address 


65 


(31.4) 


56 


(27.9) 


121 


(29.7) 


1st Address Update returned, 
nev address 


34 


(16.4) 


43 


(21 .4) 


77 


(18.9) 


"Undeliverable" located by phone 
2nd Address Update returned, 
sane address 


9 
7 


(4.3) 
(3.4) 


9 
6 


(4.5) 
(3.0) 


18 
13 


(4.4) 
(3.2) 


2nd Address Update returned, 

"CW OlUUi COQf 


17 


(8.2) 


7 


(3.5) 


24 


(5.9) 


Phone follow-up of nonresponders, 
same address 


19 


(9.2) 


23 


(11 .4) 


42 


(10.3) 


Phone follov-up of nonresponders, 
new address 


19 


(9.2) 


19 


(9.5) 


38 


(9.3) 


Total : 


170 


(82.1 ) 


163 


(81 .0) 


333 


(81 .6) 


\ddre_j Not Verified 


37 


(17.8) 


38 


(19.9) 


75 


(18.4) 
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Table 7-3 

High School and Beyond Third Follow-up Field Test 
Comparison of Address Changes for Initial and Follow-up Responders 



Letter Responders 


Number 


Percent 




Initial letter responders (N*198) 
Same address 
Nev address 


121 
77 


(61* of 
(39* of 


198) 
198) 


Follow-up letter responders (N*117) 
Same address 

per 2nd address update form 
per telephone contact 

Subtotal: 


13 
42 

55 


.47* of 


117) 


New address 

per 2nd address update form 
per telephone contact 

Subtotal : 


24 
38 
62 


(53* of 


1 17) 
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to the second request, 47 percent were at the same address, and 
53 percent had new addresses. 

Those who responded to the initial request for address 
information were more likely to be living at the same address than 
those who were reached as the result of follow-up contact. These 
findings have implications for the full-scale survey locating 
expectations and strategies which are discuseed later in this 
section. 

7.2 NLS-72 Locating Procedures and Results 

All locating for NLS-72 sample members was done by telephone 
from NORC's central office in Chicago. Telephone locating began 
in late September 1984 and continued until early December. 
Telephone interviewers began the locating process using name, 
address, and telephone- information supplied by participants at the 
time they were last contacted. The length of time since the last 
contact varied by individual from 2 to 12 years depending oji the 
last event (see list below) that a survey member participated in. 

NLS-72 Survey Events 

Base year test and survey 1972 

First follow-up survey 1973 - 1974 

Second follow-up survey 1974 - 1975 

Third follow-up survey 1976 - 1977 

Fourth follow-up survey 1979 - 1980 

Address updating 1981 

Address updating 1983 

Overall, approximately 77 percent of the field test sample 
were located (see Table 7-4). There was a considerable difference 
in the success of locating those who had participated in the 
fourth follow-up and those who had not. As Table 7-4 shows, about 
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Table 7-4 
NLS-72 Fifth Follow-up Field Test 
Telephone Locating Outcomes 



Fourth Follow-up 
Participant 

N x 600 



Located 



Same New Other 

Phone Phone Contact Total 

177 253 70 +98 

(29*) (42*) (12*) (83*) 



Not located 



Total 

102 
(17*) 



Fourth Follow-up 
Nonparticipants 

N = 97 



Located 



Same New Other 
Phone Phone Contact 



8 17 
(8*) (18*) 



1 1 
(11*) 



Not located 



' al 

36 
(37*) 



Total 

61 
(63*) 



163* 
(23*) 



83 percent of the fourth follow-up participants wee located; less 
than half (37 percent) of the nonparticipants were found* 

Field test results also indicate there were differences in 
the utility of the locating information on record from previous 
follow-ups for survey members with different participation 
histories. While close to one-third (29 percent) of the fourth 
follow-up participants were located at the phone number on record, 
less than one-tenth (8 percent) of the nonparticipants could be 
contacted at +he number they had last supplied. The participation 
histories of the individuals within the fourth follow-up 
nonparticipant group vary and, therefore, so does the age of 
locating information. For example, some survey members did not 
participate in the fourth follow-up , but did participate in the 

er|c 115 



third follow-up survey three years earlier, in 1976-1977. Others 
have not participated since the base year in 1972. Therefore, 
name and address information for the group overall was 8 to 12 
years old. 

Table 7-4 also shows differences in the success with which 
respondents can be "traced" when names, addresses, and phone 
numbers on record are no longer current. Forty-two percent of the 
fourth follow-up participants were traced and eventually contacted 
at new phone numbers, but only 18 percent of the nonparticipants 
could be found. Not only were there fewer direct links to those 
in the nonpar tic ipant group (that io, fewer were reached at the 
phone number on record), but acquiring information from 
respondents' family, friends, and acquaintances was also less 
successful. All information for the group of nonparticipants, 
including information on families, friends, and acquaintances, 
was, of course, older and more likely to be out-of -\ite. Other 
tracing means such as directory assistance and address/telephone 
number cross-reference services were also less successful because 
of the longer periods of nonparticipation. The longer the period 
of time since the last contacts with these respondents, the 
greater the number of moves (and name changes) that were possible, 
and the less likely it was to trace them successfully to their 
current location. 

Of the NLS-72 cases that were not located by telephone 
interviewers during the prefield phase, a small number were 
selected for additional locating effort by NORC field staff. The 
objective was to assess the costs and effectiveness of locating 



9 

116 



activities that vere likely to be needed for the most difficult 
cases in the NLS-72 sample. 

Twenty-four cases were chosen for actional field effort. 
They were chosen from Stratum E (see the subsection on sample 
design in Section 1, "Field Test Activities," for a full 
description of this stratum), which is composed off survey members 
who were fourth follow-up nonparticipants and who were either 
never-married parents or had experienced marital disruptions. 
Eight cases were chosen from each of three urban areas (New York, 
Chicago, and Los Angeles) that have consistently offered locating 
challenges. 

All information available for the twenty-four NLS-72 survey 
members was sent to the field managers of the areas mentioned 
above to be reviewed for any possible leads that could be pursued 
at a regional or local level. The information included 
documentation of the most recent telephone tracing done at the 
NORC central office during the fall prefield locating period as 
well as survey records of various types (e.g., tracing logs from 
previous follow-ups, address update forms) that dated as far back 
as the basa year of the survey. 

Field managers contacted state and county agencies (such as 
the Departrsnt of Motor Vehicles, the Bureau of Vital Statistics, 
and prisons) for information about survey members 1 current 
whereabouts. They also contacted local insitutions (such as 
public libraries and various municipal offices). If survey 
members had reported postsecondary education during previous 
follow-ups, school administrative offices and alumni associations 
were contacted for address information. 
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The field staff also extended the central office locating 
effort by attempting telephone contact with family numbers, 
friends, and other acquaintances identified in old records. Field 
managers indicated that for some cases there were leads in old 
records that might be pursued by actually going to respondents' 
old neighborhoods, although this was not done during the field 
test because of cost considerations. 

In all, six of the twenty-four cases were found. The 
activities of the field staff represented the next level of 
locating effort beyond telephone tracing from a central location. 
The twenty-four -case experiment indicated the level of effort and 
the associated costs required for the most difficult locating 
cases. It further indicated that even with considerable offort 
only a small proportion of these cases can be found. 

7.3 Implications of field test locating results for the 
full-sca.1e survey 

The cost and effectiveness of different locating procedures 
used during the field test varied and will be reviewed in this 
section. Although the comparison of results provides practical 
information (and perhaps some methodological insights), it is 
necessary to remember that there were significant differences in 
the characteristics of the samples for which the different methods 
were used. Furthermore, the methods that were used were chosen 
because they were expected to be the most successful ways of 
achieving particular field test objectives. With the exception of 
the HSftB mail and telephone follow-up of nonresponders, the 
purpose of using different methods was not experimentation, and 
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therefore, neither conditions nor controls appropriate for 
conducting experiments were established- A note of caution must 
also be made about extrapolating from these results to research 
efforts that do not have longitudinal designs. 

Both the costs and the effectiveness of the two follow-up 
methods differed. The initial mail request for name and address 
information resulted in responses from almost half of the HS&B 
survey members. With additional follow-up of nonresponders, 
however, 82 percent of the sample were eventually located (see 
Table 7-2, above). 

The cost of mail follow-up was about $3.00 per completed 
case. (This unit cost, which is calculated by dividing total 
expenses by the number of cases actually located, also includes 
the cost of work on cases that are not located.) Costs were 
comparatively modest because the same procedures and materials 
used for the first mailing were used for the second. Therefore, 
minimal management and preparation was required. It should be 
emphasized that neither the unit cost for mail follow-up discussed 
here nor that for telephone foilow-up have included the much 
larger startup costs required for a full locating effort on the 
main survey. The costs as they are currently calculated thus must 
be interpreted to be applicable solely to the field test. 

The cost of telephone follow-up was approximately $10.00 per 
completed case. Telephone follow-up was conducted from NC**C»s 
central office in Chicago. Costs included telephone int .rviever 
labor as well as telephone expenses, with a considerable 
proportion being long-distance interstate charges. The method was 
more effective (contact was made vith about twice as many survey 
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members), but the unit cost cf $10. 0C per completed case was over 
three times that of mail follow-up . Again, the unit cost for 
telephone follow-up does not include startup costs. 

NLS-72 locating, all of which was done by telephone, resulted 
in name and address updates for 77 percent of the NLS-72 field 
test sample. The cost of telephone locating was greater than mail 
contact without any follow-up and about the same as mail contact 
with follow-up. 

The cost for the field staff effort that was undertaken for 
the twenty-four NLS-72 cases was almost $100 per completed sase. 
This cost was in addition to the extensive telephone locating that 
was done from the NORC central office in Chicago. Given that six 
of twenty-four cases were located, the return on this additional 
effort was only 25 percent of the fielded cases. 

Although it would be desirable to use all effective locating 
methods during the prefield period for the full-scale purvey, to 
do so would require resources far beyond what has been allocated 
for this task. Telephone locating for the entire 3ample is not 
possible from a cost standpoint. Furthermore, telephone contact 
alone would not allow for the distribution of survey newsletters 
that are considered an important opportunity to renew survey 
members' interest in the study and to remind them of the 
importance of their participation. 

The cost of either mail or telephone follow-up of 
nonresponders alone is prohibitive because of the large number of 
survey members who are not be expected to respond to the initial 
locating letter. Based on the HS&B second follow-up main survey, 
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70 percent of the sample did not respond to the locating letter. 
Even at the comparatively low cost of $3 per case, the cost of 
mail follow-up for 70 percent of the combined HS4B and NLS-72 
samples (almost 27,000 survey members) would be about $81,000; tho 
cost of telephone follow-up would be approximately $27C,000. 
These costs would be in addition to that of an initial locating 
effort which is even greater because of the setup and startup 
costs of this major task. 

The field test was useful in identifying which portions of 
the sample are likely to be most difficult to locate. It was also 
valuable in indicating which locating techniques are likely to be 
most successful (or necessary) for diffsrent subgroups and at what 
costs. This information will aid in guiding the field staff who 
will continue tracing efforts at regional and local levels for 
survey members wb; are not located during the prefield period. 
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8. TEACHING SUPPLEMENT QUESTIONNAIRE 

A supplement to the NLS-7? fifth follow-up core questionnaire 
was sponsored by the National Science Foundation (NSF) to 
investigate factors that influence decisions for and against 
choosing teaching as a career. The Teaching Supplement 
questionnaire gathers information about the reasons that 
individuals acquire training to teach, why they do or do not 
pursue or continue in careers as elementary and secondary 
teachers, and how they feel about various aspects of the teaching 
field. The supplement also addresses the issue of incentives that 
might encourage people to become mathematics and science teachers. 
Additional NSF items focusing on teaching issues were included in 
the NLS-72 main questionnaire and are analyzed elsewhere (see the 
section on -Logical Consistency between Filter and Dependent 
Items"). This section discusses only the supplementary 
questionnaire. 

The leaching Supplement was sent with the NLS-72 core 
questionnaire to survey members who had been identified as 
teachers or -potential- teachers. The selection criteria for this 
group, referred to as Stratum A throughout the field test report, 
is fully discussed in the sample design subsection (1.2) of 
Section 1, -Field Test Activities.- Briefly, Stratum A members 
are those who had worked as teachers, those whose educational 
background had prepared them for teaching, or those who had 
iudicated an interest in teaching during a previous follow-up 
survey. 

The same field test procedures were used for Stratum A as for 
the other NLS-72 strata. Telephone locating was conducted during 
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the fall of 1984- and resulted in address updates for over 97 
percent of the stratum- Questionnaires were mailed during the 
first week in February 1*85 to those for whom address updates had 
been acquired, Th^ survey response rate for this stratum, 86 
percent, was the highest achieved by any of the NLS-72 strata. 

Although the response rate of Stratum A was 86 percent, only 
69 percent of the r aple returned the Teaching Supplement with the 
core questionnaire (see Table 8-1). Furthermore, not all of the 
Teaching Supplements that were sent back were filled out; 
6 percent were returned blank- About one-third of those who 
returned the supplement (21 percent of the stratum) answered only 
the first question indicating that they had neither had teaching 
exDerience nor been trained as teachers. The remaining questions 
in the supplement were not applicable to these respondents and 
were legitimately skipped. After a series of questions on 
certification, a filter question reduced the number of 
respondents who completed the rest of the supplement to 71 persons 
who were currently teachers or had been teachers formerly. 
Therefore, analysis of most of the Teaching Supplement data is 
based on the responses of these 71 survey members who constitute 
about 36 percent of the stratum. 

What appears to be a low response rate for the Teaching 
Supplement is attributable in large part to sample design 
features. The definition of teacher and "potential" teacher for 
sampling purposes was inttntionally broad anu was ^xpecxed to 
include survey members for whom the Teaching Supplement would not 
be aprlicable. One of the field test objectives was to estimate 

ERJC 2 123 



Table 8-1 
Final Status of Stratum A 
(N*200) 

(Number and percent of 200)* 



(2.5%) 
Unlocatable 
during prefieid 
stage 



20 (10*) 
Returned 
undeliverable; 
never located 



195 (97-5*) 

Questionnaires mailed 



175 (87-5*) Final dispositions 

4 (2,0*) 
Refuna'is 

33 (16,5*) 

Returned core questionnaire only 

138 (69.0*) 

Returned core questionnaire and 
Teaching Supplement 

12 (6,0*) 

Teaching Supplement blank; 
not applicable 



126 (63,0*) 

Usable questionnaires 

42 (21.0*) 

No teaching experience, 
no teacher training 

84- (4-2,0*; 

Teaching experience 
or training 

13 (6.5*) 

Never taught 

71 (35.5*) 

Current and former 
teachers 



♦The distribution of cases in tne lower right box is based on Teaching 
Supplement questionnaire responses- All other figures arc from mail and 
receipt control records- 
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the pre; :-tion of the sample who were of interest for research and 
policy analysis purposes (i.*., those who had taught in the past, 
were currently teaching, or were interested in the teacning 
profession). Another objective was to evaluate the ability of 
survey instruments ^o route those survey members into and through 
questionnaires and questionnaire sections appropriately. The 
selection criteria for inclusion in Stratum A supported these 
objectives but also resulted in what appears to be a low response 
rate for the Teaching Supplement. Table 8-1 shows how Stratum A 
cases were narrowed down to the proportion that completed Teaching 
Supplement questions. 

The performance of Teaching Supplement questionnaire items 
are examined from three perspectives: inter-item consistency, 
nonresponse, and problems discovered during coding. Inter-item 
consistency refers to whether or not respondents can follow the 
system of filter and dependent questions. Filter questions are 
generally answered by everyone and are followed up by instructions 
for some individuals to skip certain portions of the questionnaire 
and for other individuals to answer the next question in sequence. 
If instructions are not explicit or straightforward enough, 
respondents may become confused and not answer the appropriate 
questions. Inappropriate patterns of response, or inter-item 
inconsistency, were detected by comparing the number of 
respondents who answered a filter question in a given way compared 
to the number of respondents who answer the subsequent dependent 
questions. Crosstabulations of filter by dependent questions also 
were used to assess consistency. 
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Nonresponse is a signal which may indicate circumstances 
within the questionnaire that discourage respondents from 
answering certain questions. Reasons include respondent 
reluctance to answer a particular question, misunderstanding or 
overlooking of a question, or inapplicability of answer categories 
to the respondpnt. Nonresponse may also result from inappropriate 
or complicated skip patterns. 

During the coding process, it was possible to detect wording 
problems, insufficient response categories, lack of clarity about 
who should respond, and so on. Also, respondents were encouraged 
to write in comments and the coding staff recorded all such 
information pertinent to improving the questionnaire. 

In general, most items in the Teaching Supplement appeared to 
function properly and will not require revision. For one series 
of questions, virtually no response variation was found. 
Question 15 asks, "Have you attended any National Science 
Foundation-sponsored institutes, conferences, or workshops?" 
Ninety-nine percent of respondents answered "no M . In question 
15A, respondents are asked to indicate which NSF activities they 
had participated in. Only one person had attended a NSF activity 
and was able to select among these categories. Similarly, only 
one person was able to respond to parts 15B and C. Since the 
series of questions was not applicable to 99 percent of 
respondents, deleting it from the main Teaching Supplement 
questionnaire is recommended. 
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Some items were found to need modification; these items are 
reviewed below- In each case, the nature of the problem is 
discussed and a recommended action is offered. 

Item 1 : What requirements did you have to meet for graduation 
from a teacher training program and for initial 
certification? 

There is some evidence that former teachers were confused by 
this question; some teachers thought that it only applied to those 
currently in teaching jobs. To alleviate this problem, the 
supplement needs «ome general instructions which make it clear to 
former teachers how the various questions apply tc them. 

Item 3: As of the first week of February 1985, in which specific 
field(s) (e.g., art, music, health) were you certified? 
(CIRCLE ALL THAT APPLY. FOR EACH FIELD CIRCLED, ENTER 
MONTH AND YEAR CERTIFICATION WAS RECEIVED.) 

At the bottom of the choices in question 3, there is an 

instruction to skip to question 5 which is intended only for 

someone who chooses the category "Not currently certified." The 

question format could imply that all respondents should skip to 

question 5. This may explain why only 69 out of 84 respondents 

answered question 4. This item will be examined to determine the 

best placement of this instruction so that only respondents "not 

currently certified" will skip to question 5. 



Item 5: Did ycu teach during the 1984-85 school year? 
(CIRCLE ONE) 

Yes..> (GO to 5B) 1 

No, but taught before (ANSWER 5A, B and C 2 

No, have never taught (STOP HERE 3 

THANK YOU FOR YOUR PARTICIPATION. PLEASE RETURN 
THIS QUESTIONNAIRE ALONG WITH THE MAIN QUESTIONNAIRE 
IN THE ENVELOPE PROVIDED) 
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This question is the second filter question; it eliminates 
individuals who did not have any teaching experience. Because of 
this question, the number of respondents dropped to 71 since 13 
individual* replied that they had never taught. After examination 
of the crosstabulations, it was found that with the exception of 5 
respondents who incorrectly answered 5A as well as 5B, there wer- 
no problems or ambiguities with this question. 

Item 7: What subject areas or grade levels have you taught in 
the last two years (or the last two years that you 
taught)? (WRITE IN SUBJECT BELOW, THEN ENTER THE 
SUBJECT CODE FROM LIST ON PAGE 5.) 

Some confusion was apparent among respondents answering 
Questions 7A-2, 3, and 7B-2 and 3, They were asked to enter "99" 
if the categories did not apply to them in Question 7A f but were 
then asked to enter "0" if the categories did not apply to them in 
7B, Rather than use two different codes for the same response, it 
would be preferable to change the instructions so that respondents 
can enter "0" in both questions for simplicity ?nd consistency. 

Part A of Question 7 instructs the respondent to enter the 
subject or grade level of the first, second, and third most 
frequently taught courses, and a code for these subjects/grades, 
A list of these codes is found on page 5 of the questionnaire. 
Some respondents tried to give both subject and grade level codes 
for each part of the question when they were only required to give 
either a subject or a grade level code. The question was intended 
for elementary teachers to choose the appropriate grade level ccie 
and for secondary teachers to choose a subject code but both 
groups may have felt the need to give both types of information. 
The question will be revised in order to clarify the instructions. 
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Item 9: Are you currently required to teach any courses for which 
you do not feel adequately qualified? 

There were some former teachers who felt awkward answering 

this question because it is written in the present tense; teachers 

indicated that the question implied that they could not answer the 

question oased on previous teaching experience. The question 

could be re-worded "Have you ever been required to teach any 

courses for which you did not feel adequately qualified?" 

Item 11: What is the ability makeup of your current (most recent) 
classes? 

At the time that approval was granted for the field test of 
the Teaching Supplement, OMB expressed concern about whetbur 
respondents would be able to choose among the answer categories to 
describe the ability level of their current or most recent 
classes. The frequencies of field test data show that there was a 
spread of responses among answer categories: 13 percent reported 
classes with high-ability students, 19 percent reported low- 
ability classes, 22 percent reported average-ability classes, and 
48 percent reported classes of students of widely differing 
ability levels* Based on this distribution and a high response 
rate (90 percent), no changes are recommended for this question. 

Item 13: In some schools or school districts where there are 

teacher shortages in certain subject areas, especially 
math and science, teachers are urged to obtain retraining 
in another area of expertise, to help meet the shortage. 
In the past two years, has your school or school 
district urged retraining of teachers in order to meet 
teacher shortages? 

Onl/ 67 of 8* respondents answered this question. Use of the 
present tense may have confused former teachers but this 
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discrepancy can be rectified by rewriting the question to read "In 
your experience as a teacher, has your school or school district 
urged retraining of teachers in order to meet teacher shortages?" 
This present tense wording pattern found throughout the 
questionnaire continued to cause confusion for former teachers. 
Each question should be worded so that former teachers are 
explicitly instructed to answer the question, or are provided with 
a response option to indicate that they are not currently 
teaching. 

Item 26: How would you describe the location of your school? 

OMB also had concerns about whether respondents would be able 
to choose among answer categories to describe the location of 
their school. Again, there was good response variation: 22 
percent said that their school was within the central city, 9 
percent said outside the central city but within a metropolitan 
area, 20 percent said in a suburban area, 34 percent said in a 
small town, and 14 percent said in a rural area. Based on the 
variety of responses and high response rate (90 percent), no 
changes are recommended for this question. 

Item 30: To what extent does your school provide resources to 
help its "effective school" project to be successful? 

Seventeen people should have answered this question based on 

the filter question in item 28 instructing respondents who respond 

"yes" to skip to questions 29 and 30. The response rate to 

question 29 was satisfactory. However, at question 30, the 

correct number of respondents is problematic. For example, 

question 30A has a sample size of 13 and a missing value of 4, but 
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30C and D have a sample size of 18 and a missing value of 4. This 
indicates that 5 people followed an incorrect skip pattern either 
from question 28 or 29. Perhaps the question can be prefaced with 
"(FOR CURRENT TEACHERS ONLY)." 

At the end of the Teaching Supplement, respondents were asked 
to indicate the number of minutes required to complete the 
questionnaire. The mean number of minutes was 27. Table 8-2 
displays the distribution of participants across the time 
segments. The target length of time for the Teaching Supplement 
was 15 - 20 minutes. Less than half of the respondents completed 
the questionnaire within the target range. Nineteen percent took 
between 20 and 30 minutes, while another 26 percent took over 30 
minutes. 



Table 8-2 

Minutes Required to Complete the Field Test 
Teaching Supplement Questionnaire 





Number of 




Cumulative 


Minutes 


Respondents 


Percent 


Percent 


10-20 


30 


45 


45 


21-30 


19 


?9 


74 


31-40 


8 


12 


86 


41-50 


7 


1 1 


97 


over 5 1 


2 


3 


!00 



Although the Teaching Supplement now exceeds the respondent 
burden initially proposed, there is no clear evidence that the 
burden level has a dramatic effect on survey members 1 willingness 
to complete the main or supplementary questionnaires . 
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The size of the field test supplement (including both the 
main questionnaire items and those in the supplementary 
questionnaire) poses more of a problem for editing, coding and 
other data processing costs than for respondent burden and data 
collection costs. Since there is a second supplement focusing on 
teaching issues currently under development that will help to bear 
the processing costs (and will also contribute an additional 
incentive payment to all respondents who are sent a supplementary 
questionnaire), we do not believe that major reductions in the 
supplement should be made prior to fielding the survey. However, 
continuing efforts will be made to identify items that add little 
descriptive or analytical power to the dataret jr that obtain too 
little response variation to be analytically useful. 
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